Virtual memory is a construct used to provide separate memory spaces for different process, i.e. virtual memory allows each program to have an independent view of the memory space that is not restricted by other programs or the actual size of physical memory.
For each process, there is a mapping from its perceived address space to actual locations in physical memory and/or some other device, usually a hard drive. Every time a process accesses memory, a lookup is done to find the actual location of the data. If the data is in physical memory, this memory is accessed. Otherwise, the operating system takes over and either deals with moving the data to physical memory or signals an error (e.g. if a bad memory address was used).
The hard drive is essentially used a very large but slow of memory cached by RAM. This caching process leads to only active portions of a process staying ``resident'' in RAM (see the output of the top program on a Unix machine for an example). A process running on top of this system is unaware of this process since the operating system deals with all the details of moving data back and forth. Previously, this was left to programmers in the form of overlays (used as recently as MS-DOS).
Inside the virtual memory system, the operating system divides the memory space into blocks called ``pages''. Pages are aligned blocks whose size is a power of two bytes. All pages have the same size so a page is uniquely identified by its high order bits. Given a virtual memory address, the higher order bits are extracted to get the index of the virtual page which is used to lookup the index of the real page in the page table. When no entry is found in the page table, it is called a page fault and the virtual memory hardware calls an operating system function (software) to deal with it. If the virtual page exists but is on the hard drive, the operating system deals with copying the page to real memory and adding the appropriate entries in the page table. Otherwise, an error occurs (usually a segmentation violation). As pages are rearranged by the operating system, the user process is unaware of the underlying arrangement - the pages of a process may be moved and are not necessarily continuous in real memory.
Given the high cost of accessing a hard drive, pages are large to try and amortize the cost of a page fault over many memory accesses and considerable effort is spent in software to minimize page faults. The page table is fully associative and fancy software algorithms are used to minimize the number of page faults. Additionally, virtual memory systems are always write-back, sometimes called copy-back.
The page table is located in main memory and is pointed to by a special register. Each process has its own page table. As a page table can be fairly large, various tricks are used to keep the memory footprint small such as growing it incrementally and using multiple levels. One common point to these techniques is that some of the page table must be locked into memory to avoid infinite page faults trying to access the page table. In addition, there is also a cache for the table called the translation lookahead buffer (TLB).