Jim Dettman
jimdettman at verizon.net
Thu Aug 31 13:17:03 CDT 2006
John, <<All of my machines are 64 bit (AMD) processors so theoretically they can use more than 4 gig but how?>> A. The OS must support it. B. Executables must be compiled for it. Where all the confusion comes in is in talking about address space vs memory. They are two separate things and physical memory is not utilized anything like your thinking. For a given processor / OS, there is a maximum amount of memory that can be addressed by any given process. This is called the virtual address space. The OS in order to run the process takes up some of that and is mapped into that virtual address space (it usually takes the upper half and there is only one copy mapped into each processes address space). To add to this, some of the space assigned for the process is also used for the OS. Some of that is swappable (can be moved to the swap file it needed) or resident (must always stay in physical memory). Now it gets a little complicated. Each process is given a working set, that is a number of memory pages it can use at any given time. In addition to that there is a page frame table. This is a list of pages that a process is using and they are mapped into the virtual address space. In other words a program *thinks* it can address 4GB of space, but until it wants it, it's not tracked. For each page table entry, the OS keeps track of the virtual address space that it's mapped to and if the page is in the current working set, on the modified page or free page lists, or if it has been paged out to disk. For the working sets, most OS's have a couple different settings that can be played with: Working set default. Working set quota. Working set Extent. Working set increment. Working set decrement. To keep track of physical memory, there are two other lists in the OS: Free page list Modified page list The free page list is use to keep track of physical pages that are available for use by any process. The modified list are physical pages that have been modified by a process, but have not yet been written out to the page file (I'll get to this in a minute). They also have some internal settings to control page faulting. When a process is created, it starts at the default working set size. As the process needs more memory (ie. uses a variable) the OS adds an entry to the page frame table, grabs a page of memory from the free page list, and assigns it to the working set. Once the processes working set is used up, the OS sees if it can be expanded. If so, it expands by the increment. It does this until it reaches the working set extent. If a process still needs more memory, it needs to give up a page in it's working set. The OS takes one of the working set pages, writes it to the modified page list if it's been changed or the free page list if it has not (it's written to the bottom of the list), then grabs a new page from the from the top of the free page list and puts it in the working set. This process continues on. If a process no longer needs the memory, it's released from the working set and put back on the free page list. In the background as part of the OS is a process called the swapper, which controls all this. It monitors the free page list. If not enough free memory is available, it takes the pages from the modified page list, writes them to the page file on disk, and then moves those pages to the bottom of the free page list. What happens if a process needs a page back that it gave up from it's working set? First the OS checks if it is on either the free or modified page lists. If so, this is called a soft fault and the OS can easily move the page back into the working set. If however it was written to the page file on disk (because the OS ran low on physical memory), it's a hard fault. Last, if the OS runs really low on memory, it can elect to swap out a process and it's entire working set. It will do this only if: 1. The modified page list has been flushed. 2. All processes working sets have been trimmed back to the working set quota. Swapping is expensive and done only as a last resort. A process will also swap out if the overall number or processes allowed to be memory resident at one time has been exceeded. I haven't explained everything, like balance set slots, etc. But that's the way virtual memory works in a nutshell. It varies by the OS, but that's been the model used by VMS, NT, XP, etc. Jim. -----Original Message----- From: accessd-bounces at databaseadvisors.com [mailto:accessd-bounces at databaseadvisors.com] On Behalf Of JWColby Sent: Thursday, August 31, 2006 10:05 AM To: 'Access Developers discussion and problem solving'; dba-sqlserver at databaseadvisors.com; Tech - Database Advisors Inc. Subject: [AccessD] Using 4 gbytes of RAM Is anyone using a machine with more than two gig of Ram? I am reading things about doing this and there seems to be a lot of confusion on the matter. Much of the confusion seems to come from the fact that until the 64 bit versions of the X86 processors, the "internals" of the processor were all 32 bits which limited direct generation of addresses to 4 gig. All of my machines are 64 bit (AMD) processors so theoretically they can use more than 4 gig but how? Everyone seems to agree that in XP32, only 4 gig is available to work with and that things like video cards and other "machine stuff" eats into that. Thus regardless of anything, you will end up with less than 4 gig due to that, perhaps as little as 3.25 gig. After that, confusion reigns, at least in my mind. Understand that I am discussing Windows XP 32 specifically, although anyone is welcome to chip in what they "know" regarding other windows versions as long as they specify what version they are discussing. The next thing that is discussed is that the OS itself reserves 2 gig for itself (which includes the video etc AFAICT), leaving up to 2 gig for EACH APPLICATION, implying that any application can have / use up to 2 gig. Some claim that if you have 4 gig, the swap file won't be used since that is part of the 4 gig available. It seems illogical since each application can use up to 2 gig. However I have created a separate partition to hold my swap file and tried to create one of 6 gig and Windows refused, limiting the size of the swap file to 4 gig. So I am trying to figure out what the reality is. If a machine has 4 gig physical ram, and "machine stuff" uses (for argument's sake) .75 gig, is the .75 gig of physical ram simply unused? Is it used by the OS for buffers? What about the swap file? If you have a 4 gig swap file, would you have 7.25 "total available memory" for Windows use? Does this indeed then limit the APPLICATIONS to 5.25 gig of "total" memory and 2 gig of physical memory? And what happens if you manage to get (for the sake of argument) 8 gig stuffed into a machine. Some motherboards claim to handle that much but can Windows XP 32 actually use it? Can Windows XP 64? If you need this much memory is Windows XP 64 a way to get at the memory? If you are still using 32 bit apps, will it actually map as much as 2 gig of physical memory to the 32 bit apps so that (for example) 3 high powered memory hungry 32 bit apps actually had 2 gig available to each of them all the time? If anyone has run across any authoritive (on-line) reading on the subject I would love to look at it. John W. Colby Colby Consulting www.ColbyConsulting.com -- AccessD mailing list AccessD at databaseadvisors.com http://databaseadvisors.com/mailman/listinfo/accessd Website: http://www.databaseadvisors.com