Re: [Hampshire] 4GB on AMD64

On Sun, Jan 03, 2010 at 04:41:00PM +0000, Samuel Penn wrote:
>
> Hi all,
>
> Does anyone have any ideas what needs to be done to get Linux to
> recognise 4GB of RAM? I've got a box running 64bit Gentoo Linux (AMD),
> which I've just upgraded to 4GB, however Linux is only showing 3GB
> available (according to top).
>
> The machine's BIOS is displaying 4GB installed, it's just Linux that
> only seems to see 3GB.

First thing from the brown-paper-bag list: are you certain you're
running a 64-bit kernel? Check with uname -a

The next thing is: is your BIOS capable of running with a memory
hole at the 3GiB mark? If so, enable that.

Your processor has a large (40-bit) address space, into which must
be mapped not only your RAM, but also certain chunks of your hardware,
including all of your PCI and PCI-E devices. These devices are
typically mapped with no address translation, so if your graphics card
has been configured to use a PCI address of, say, 0x00e0000000, then
the address space from that point onwards can't be used for mapping
RAM into. (The MMU built into every x86 processor from the 386 onwards
handles very flexible RAM mapping). Now, with less than 4GiB of RAM,
this isn't usually a problem, because the machine can map the RAM from
0x0000000000 to 0x007fffffff, and configure the PCI devices to appear
at addresses above that range. With 4GiB of RAM, the RAM is normally
mapped in from 0x0000000000 to 0x00ffffffff. So, you'd think that the
PCI devices would get mapped above that range -- except that they
can't: PCI uses a 32-bit address space, so PCI devices can't be
mapped higher than that.

So, what most BIOSes do is simply hide the range of RAM from
0x00c0000000 to 0x00ffffffff, and map the PCI bus into there
instead. This loses you 1GiB of RAM (anything above that range is
still seen). Some BIOSes have an option to put a "hole" in the memory
allocation at that range, and make the top 1GiB of RAM appear from
0x0100000000 to 0x013ffffffff, leaving 1GiB of free address space to
put the PCI bus in. Enable the option, and your RAM will be visible to
Linux again.

Some high-end hardware has an IOMMU, which is a device that handles
address-space translation for the PCI bus in the same way that the MMU
handles translation for RAM. These machines can map PCI devices
anywhere in their address space, and don't have this problem. (They
can also do nice/clever things with virtualisation, but that's kind of
irrelevant here).

Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
  --- "I used to do this as a kid" "... and now we get paid for it" ---