[Linux][Memory] User/Kernel Space

--------------------------------------
Q: How does Linux partition logical address space as far as User/Kernel space is concerned, especially when it comes to deciding what gets mapped to virtual versus physical RAM? - Questions
--------------------------------------
Source: http://fixunix.com/questions/14616-how-does-linux-partition-logical-address-space-far-user-kernel-space-concerned-especially-when-comes-deciding-what-gets-mapped-virtual-versus-physical-ram.html

The norm for a memory allocation scheme is to use physical memory before
dumping the overflow to disk. However with Linux it's unique in that
there's a set limit for user and kernel space (aka monolithic kernel).
Which leads me to believe that the logical address space is set aside
ahead of time. Meaning that if a person has 2GBs of physical RAM and
chooses option 3GBs/1GBS of user/kernel space, that either 1GB of kernel
space and 1GB of user space will be allocated to physical RAM; OR, 2 GBs
of user space will go to physical RAM while the remaining 2 GBs (1 user,
the other kernel) will be virtual. This seems rather asinine, so I
hardly believe this is how it works.

It was also proposed to me that the system only considers how much it -
will- give to either user or kernel space. Meaning that at boot up, if a
system only uses 2 MBs of Kernel space, only 2 MBs gets mapped from the
the total available memory for Kernel space (as there's a cap). Also
meaning that those first 2 MBs would be allocated to physical memory.
Then as more would be requested more would be allocated for either user
or kernel space, straight from physical memory, till it ran out. This
sounds like a more solid schema, but at the same time it seems silly to
do things on a first come, first serve basis. It seems more reasonable
that it would partition each application such that each has some
physical memory to work in, and then to use a memory map for anything
additional to disk.

So the question is, how does Linux delegate it's memory with regards to
user/kernel space? Does Linux treat User/Kernel space just as a hard
limit, allocating memory as it's needed until it hits the cap for a
particular "type" of memory? Or does it preordain that kernel space will
be in physical memory while user space will get the remainder, whether
it be physical or not? Is there possibly another partitioning scheme,
similar to the one I suggested above?

I'm beginning to believe it's a simple tallying scheme, checking to see
how much kernel space or user space has been allocated.

ie/
3GB/1GB of User/Kernel space is available.
2 MBs gets allocated at boot-time for the kernel and it's modules -
(1.000GBs-0.002GBs=NewAvailableKernelSpace)
50 MBs gets allocated to X in user space - (3.000GBs-
0.050GBs=NewAvailableUserSpace)
So on and so forth.

I imagine that the logical address space is just a series of pointers
telling the system where everything is. For example, the first logical
address might point to physical memory for kernel space, while the
second logical address unit might point to user space in virtual memory
(similar to the example above). I imagine that's the whole value of
logical address space - it provides the HAL.

Any reference material or solid answers surrounding the mysteries of
Linux's memory allocation would be appreciated.

Thanks,
Dustin
--------------------------------------

--------------------------------------
Q: Virtual memory - user space kernel space
--------------------------------------
Source: http://fixunix.com/linux/7872-virtual-memory-user-space-kernel-space.html

Hi,
I m having some questions in Linux memory management.
I m considering about 32 bit machine only.
If you have a 512 MB of memory, how it is divided between kernel space
and user space?

What is the link between this division and 3GB userspace and 1 GB
kernel space division.
Is the 3G/1G division is for virtual memory?
Is the total Virtual memory can be 4G only? I mean is it the maximum?

Thanks in advance.

regards
SaranJothy

Re: Virtual memory - user space kernel space
SaranJothy wrote:
> Hi,
> I m having some questions in Linux memory management.
> I m considering about 32 bit machine only.
> If you have a 512 MB of memory, how it is divided between kernel space
> and user space?

A (relatively) small part is statically allocated to hold the kernel 
code and static data. The rest is dynamically allocated to whatever is 
required (user, drivers, kernel, ...)

> What is the link between this division and 3GB userspace and 1 GB
> kernel space division.

None.

> Is the 3G/1G division is for virtual memory?

Yes.

> Is the total Virtual memory can be 4G only? I mean is it the maximum?

In 32 bit Linux: yes, you cannot address more than 4GB of virtual 
addresses with 32 bit pointers.

-- 
Josef Möllers (Pinguinpfleger bei FSC)
If failure had no penalty success would not be a prize
-- T. Pratchett


Re: Virtual memory - user space kernel space
On Feb 8, 1:45 pm, "SaranJothy" wrote:
> Hi,
> I m having some questions in Linux memory management.
> I m considering about 32 bit machine only.
> If you have a 512 MB of memory, how it is divided between kernel space
> and user space?

you bumble virtual and physical memory. kernel divides _only_ virtual
memory between kernel space and user space,
physical memory are share to both. so, if for example, I want to
allocate n mb from user space, i can get some virtual address p
associated with physical page g.
and if i after it will try to allocate n mb from kernel space i can
get some virtual address i, associated with the same physical page g.
it's normal.

>
> What is the link between this division and 3GB userspace and 1 GB
> kernel space division.
> Is the 3G/1G division is for virtual memory?

yep

> Is the total Virtual memory can be 4G only? I mean is it the maximum?
>

nope. intel, for example, support PAE feature, that performs 64G
virtual space.

> Thanks in advance.
>
> regards
> SaranJothy


Re: Virtual memory - user space kernel space
Thanks for the reply.
As said,
> A (relatively) small part is statically allocated to hold the kernel
> code and static data. The rest is dynamically allocated to whatever is
> required (user, drivers, kernel, ...)
So, It depends upon the total size of kernel code and static data.
Right?

I read like 1G is allocated for kernel and 3G is allocated for
userspace in Virtual memory.
So part of RAM is mapped to 1G Kernel space address.
If part of RAM where kernel resides is smaller(as in most cases) than
1G, how it is mapped to 1G?

how 3G /1G break is maintained for secondary storage having just
2 G of memory?

I m having some problems in co-relating Physical memory and VM. pls.,
correct me.

Regards,
Saran


Re: Virtual memory - user space kernel space
Hi,
Thanks for the replies.
As said,
> A (relatively) small part is statically allocated to hold the kernel
> code and static data. The rest is dynamically allocated to whatever is
> required (user, drivers, kernel, ...)

So, It depends upon the total size of kernel code and static data.
Right?


1G/3G division in virutal memory is nothing but a set of pointers to
have VM management.
Set of pointers for Kernel. -1G
Set of pointers for User space - 3G
Kernel space pointers are special such that they have direct mapping
to Physical memory.
User space pointers has to go via MMU to get the location in physical
memory. Just correct me with the above understanding?


The process (User space) memory is swapped in and out to Secondary
storage to meet memory requirements of new process required to
execute.
What happens when we dont have enough Secondary storage.
say we have 2G RAM and 1.5G application is currently running.(I hope
this is possible) We have 1G of secondary storage only.
Now a 1.5G application wants to run.
(How the currently executing 1.5G application gets swapped out?

My intention of qn. is what happens when we dont have enough Secondary
memory.

Thanks for your replies,
Saran

Re: Virtual memory - user space kernel space
"just.asgard" wrote in
news:1170938227.135204.298340@p10g2000cwp.googlegr oups.com:

>> Is the total Virtual memory can be 4G only? I mean is it the maximum?
>>
>
> nope. intel, for example, support PAE feature, that performs 64G
> virtual space.

This is incorrect. PAE provides 64G of _physical_ address space. It
doesn't change the 4GB of virtual address space. You still only have 32-
bit registers.

GH

Re: Virtual memory - user space kernel space
SaranJothy wrote:
> Thanks for the reply.
> As said,
> 
>>A (relatively) small part is statically allocated to hold the kernel
>>code and static data. The rest is dynamically allocated to whatever is
>>required (user, drivers, kernel, ...)
>
> So, It depends upon the total size of kernel code and static data.
> Right?

Yes.

In the old days, all physical memory was mapped 1:1 to virtual addresses 
0xc0000000 upwards, so the kernel could access everything directly. If 
you had more than 1GB of physical memory, you could recompile the kernel 
to start at 0x80000000 or even (correct me if I'm wrong) 0x40000000, to 
accomodate up to 2 or 3 GB of physical memory.
Later, Hugemem and friends were introduced and you could have more than 
1GB of physical memory.

> I read like 1G is allocated for kernel and 3G is allocated for
> userspace in Virtual memory.
> So part of RAM is mapped to 1G Kernel space address.

Yes, the part which holds the kernel code and static data is mapped to 
the 1G kernel address space. Dynamically allocated kernel memory is also 
mapped to the space. Other pages (user pages, buffers, ...) can be 
mapped on demand.

> If part of RAM where kernel resides is smaller(as in most cases) than
> 1G, how it is mapped to 1G?

In general, there is no distinction between less than 1GB and more than 
1GB of physical memory.

> how 3G /1G break is maintained for secondary storage having just
> 2 G of memory?

??? "secondary storage" ???
You mean swap space? Linux uses swap space in addition to physical 
memory, so having 1GB of physical memory and 2GB of swap space would 
allow you to use 3GB of virtual memory, distributed amongst the kernel 
(a relatively small portion, locked to physical memory) and user processes.

> I m having some problems in co-relating Physical memory and VM. pls.,
> correct me.

Try to find a decent book on virtual memory.

An attempt of a very short explanation:
virtual memory is subdivided into "page frames", in ia32 these are 4KB.
Likewise, physical memory is subdivided into "pages", again 4KB in size.
There is a piece of hardware, the Memory Management Unit (MMU) which 
takes a virtual address coming out of the CPU and replaces this by a 
physical address, in the simplest case by a table-lookup: The upper 20 
bits (the page frame number) of a 32-bit virtual address selects one of 
the 1048576 slots in the table. This slot contains a bit "valid" and the 
upper 20 bits (the page number) of the physical address. If the "valid" 
bit is 0, then this virtual address is invalid and the access is aborted 
(SIGSEGV). If the "valid" bit is 1, the 20 bits "page frame number" is 
replaced by the 20 bits "page number",keeping the lower 12 bits of the 
(virtual) address. That 32 bit value is then used as the physical address.
When switching between processes, the kernel has to replace the low 
786432 entries in this table by the low 786432 entries of the new 
process' table. Also, you need two tables: one for kernel and one for 
the currently active user process, the kernel's low 786432 entries are 
identical to the user process' low 786432 entries. The upper 262144 of a 
user process' table _always_ have the "valid" bit cleared.

Does this help?

Josef
-- 
Josef Möllers (Pinguinpfleger bei FSC)
If failure had no penalty success would not be a prize
-- T. Pratchett

Re: Virtual memory - user space kernel space
SaranJothy wrote:

> Thanks for the reply.
> As said,
>> A (relatively) small part is statically allocated to hold the kernel
>> code and static data. The rest is dynamically allocated to whatever is
>> required (user, drivers, kernel, ...)
> So, It depends upon the total size of kernel code and static data.
> Right?
>
> I read like 1G is allocated for kernel and 3G is allocated for
> userspace in Virtual memory.
> So part of RAM is mapped to 1G Kernel space address.

If you have less than 1G of physical RAM, *all* physical RAM
is mapped into the top 1G kernel space.

The key is to understand that you can have more than one
virtual mapping for a physical RAM page. So, when userspace
requires a memory page, one of the already mapped (but unused)
pages in kernel space gets mapped into userspace as well.

> If part of RAM where kernel resides is smaller(as in most cases) than
> 1G, how it is mapped to 1G?

The kernel is typically loaded at the beginning of the physical RAM.
This, like all other RAM is then mapped to start at the 3GB boundary.

There is one exception - If you have 1GB or more of physical RAM,
the RAM which cannot be mapped into the top 1GB, gets managed as
high memory and only mapped on request.

>
> how 3G /1G break is maintained for secondary storage having just
> 2 G of memory?

It's unrelated. Of course, if you want to memory map a file, you
have to stay below the free virtual space.

Regards,

Iwo

Re: Virtual memory - user space kernel space
On Thu, 08 Feb 2007 14:06:26 +0000 Iwo Mergler wrote:

| The key is to understand that you can have more than one
| virtual mapping for a physical RAM page. So, when userspace
| requires a memory page, one of the already mapped (but unused)
| pages in kernel space gets mapped into userspace as well.

That's a feature used by VRB to make a ring buffer (provided the size
is an exact multiple of page size) that automatically wraps around
by having a mirror image mapping of the entire buffer immediately
following the first mapping in VM. That allows direct access via
tools that don't understand the ring buffer wraparound (e.g. strXXX
functions in stdlib or others) while avoiding any copy activities
to maintain both the data and empty space as contiguous areas.

See: http://vrb.slashusr.org/
--------------------------------------

-----
Cheers,
June