How to convert virtual address to physical address and revers e?

[Thekkedath Gopakumar-agt026]

I have problems about converting virtual address into physical address and
reverse. I supply a kernel function to grab cr3 value, but how do I READ
the content from the address located by cr3 to load the GDT and access the
entry in it ?

CR3 register holds the physical address of current process's PGD. It is the
GDTR
register which holds the address of GDT.

static unsigned long v2p(unsigned long va)
{
pgd_t *pgd;
pmd_t *pmd;
pte_t *ptep, pte;
struct page *page;
unsigned long pa;
pgd = pgd_offset(tsk->mm, va); /* what is tsk->mm */
pmd = pmd_offset(pgd,va);
pte = pte_offset(pte,va);
pte = *ptep;
page = pte_page(pte); /* aha, we have the pa of the page */
pa = (pte_val(*ptep) & PTE_PHYS_MASK) | (va&(PAGE_SIZE-1);
return pa;
}

The above function traverses the Page Directory and Page Table to get the
physical
address, but I do not know how safe it will be if you supply a non existant
address
to it.

Linux kernel provides you with functions like virt_to_phys and friends to
convert
a kernel mode virtual address to physical address and vice versa.

For your case, may be after you get the physical address of GDT from GDTR,
you can use 'ioremap' to get the mapped kernel mode virtual address and
access
GDTR entries with it. Do note that the code you are writing is extremely
prone to
corrupt the system (if you alter the GDT values). I am assuming that you
are
doing this for getting a better understanding of the system.




--
Kernelnewbies: Help each other learn about the Linux kernel.
Archive: http://mail.nl.linux.org/kernelnewbies/
FAQ: http://kernelnewbies.org/faq/

[Devesh Agrawal]

Hi,
I found an exellent reference on understanding the Linux MM module, I
am sure many of you will be knowing of it. Check out Bruce Perens'
Understanding the linux virtual memory manager. Available free for
download at http://kerneltrap.org/node/3790
I have never read it much hence am unable to reply to the original poster.
Regards,


On Mon, 28 Mar 2005 10:53:30 +0530, Thekkedath Gopakumar-agt026

Post by Thekkedath Gopakumar-agt026

I have problems about converting virtual address into physical address and
reverse. I supply a kernel function to grab cr3 value, but how do I READ
the content from the address located by cr3 to load the GDT and access the
entry in it ?

CR3 register holds the physical address of current process's PGD. It is the
GDTR
register which holds the address of GDT.

static unsigned long v2p(unsigned long va)
{
pgd_t *pgd;
pmd_t *pmd;
pte_t *ptep, pte;
struct page *page;
unsigned long pa;
pgd = pgd_offset(tsk->mm, va); /* what is tsk->mm */
pmd = pmd_offset(pgd,va);
pte = pte_offset(pte,va);
pte = *ptep;
page = pte_page(pte); /* aha, we have the pa of the page */
pa = (pte_val(*ptep) & PTE_PHYS_MASK) | (va&(PAGE_SIZE-1);
return pa;
}

The above function traverses the Page Directory and Page Table to get the
physical
address, but I do not know how safe it will be if you supply a non existant
address
to it.
Linux kernel provides you with functions like virt_to_phys and friends to
convert
a kernel mode virtual address to physical address and vice versa.
For your case, may be after you get the physical address of GDT from GDTR,
you can use 'ioremap' to get the mapped kernel mode virtual address and
access
GDTR entries with it. Do note that the code you are writing is extremely
prone to
corrupt the system (if you alter the GDT values). I am assuming that you
are
doing this for getting a better understanding of the system.
--
Kernelnewbies: Help each other learn about the Linux kernel.
Archive: http://mail.nl.linux.org/kernelnewbies/
FAQ: http://kernelnewbies.org/faq/

--
Devesh Agrawal
==============
The program is absolutely right;
therefore the computer must be wrong.
-----------------
How an engineer writes a program:
Start by debugging an empty file...
----------------------
Systems programmers are the high priests of a low cult. - R. S. Barton

--
Kernelnewbies: Help each other learn about the Linux kernel.
Archive: http://mail.nl.linux.org/kernelnewbies/
FAQ: http://kernelnewbies.org/faq/

[Gaurav Dhiman]

On Mon, 28 Mar 2005 10:53:30 +0530, Thekkedath Gopakumar-agt026

Post by Thekkedath Gopakumar-agt026

I have problems about converting virtual address into physical address and
reverse. I supply a kernel function to grab cr3 value, but how do I READ
the content from the address located by cr3 to load the GDT and access the
entry in it ?

CR3 register holds the physical address of current process's PGD. It is the
GDTR
register which holds the address of GDT.

static unsigned long v2p(unsigned long va)
{
pgd_t *pgd;
pmd_t *pmd;
pte_t *ptep, pte;
struct page *page;
unsigned long pa;
pgd = pgd_offset(tsk->mm, va); /* what is tsk->mm */
pmd = pmd_offset(pgd,va);
pte = pte_offset(pte,va);
pte = *ptep;
page = pte_page(pte); /* aha, we have the pa of the page */
pa = (pte_val(*ptep) & PTE_PHYS_MASK) | (va&(PAGE_SIZE-1);
return pa;
}

The above function traverses the Page Directory and Page Table to get the
physical
address, but I do not know how safe it will be if you supply a non existant
address
to it.
Linux kernel provides you with functions like virt_to_phys and friends to
convert
a kernel mode virtual address to physical address and vice versa.

The functions you are talking are specifically for converting the
physical addresses to kernel virtual address not to user virtual
address space, for convcerting the addresses to user address space, I
think we need to traverse the PGD and related structures as shown in
the code in earlier mail.

As I looked into the functions "virt_to_phys" and "phys_to_virt" in
page.h file, i found the following macros:

#define __PAGE_OFFSET (0xC0000000UL)
#define PAGE_OFFSET ((unsigned long)__PAGE_OFFSET)

#define __pa(x) ((unsigned long)(x)-PAGE_OFFSET)
#define __va(x) ((void *)((unsigned long)(x)+PAGE_OFFSET))

Actually virt_to_phys() and phys_to_virt() functions are as follows:

static inline unsigned long virt_to_phys(volatile void * address)
{
return __pa(address);
}

static inline void * phys_to_virt(unsigned long address)
{
return __va(address);
}

As per my understanding PAGE_OFFSET is the starting kernel virtual
address, that means kernel virtual address will start from
PAGE_OFFSET.

My question is, why we are subtracting PAGE_OFFSET from a given
address in __pa() macro, is it because physically kernel code is
placed in lower memory. If this is not the reason, can someone please
clarify this ?

Post by Thekkedath Gopakumar-agt026
For your case, may be after you get the physical address of GDT from GDTR,
you can use 'ioremap' to get the mapped kernel mode virtual address and
access
GDTR entries with it. Do note that the code you are writing is extremely
prone to
corrupt the system (if you alter the GDT values). I am assuming that you
are
doing this for getting a better understanding of the system.
--
Kernelnewbies: Help each other learn about the Linux kernel.
Archive: http://mail.nl.linux.org/kernelnewbies/
FAQ: http://kernelnewbies.org/faq/

--
Kernelnewbies: Help each other learn about the Linux kernel.
Archive: http://mail.nl.linux.org/kernelnewbies/
FAQ: http://kernelnewbies.org/faq/

[P.F. Tsai]

the first 3GB in VA is left for process itself, and the fourth gigabyte is
left for kernel.
during the kernel booting, va from PAGE_OFFSET was DIRECTLY mapped to pa
from 0 and thats why we subtract PAGE_OFFSET. but that was constrained for
RAM size belows 896 MB.
I am instrested in how it works in the case of RAM size larger than 896 MB
(and less than 4GB)? ram window?
could anyone give me a detail about it ?

Post by Gaurav Dhiman
As per my understanding PAGE_OFFSET is the starting kernel virtual
address, that means kernel virtual address will start from
PAGE_OFFSET.
My question is, why we are subtracting PAGE_OFFSET from a given
address in __pa() macro, is it because physically kernel code is
placed in lower memory. If this is not the reason, can someone please
clarify this ?

--
Kernelnewbies: Help each other learn about the Linux kernel.
Archive: http://mail.nl.linux.org/kernelnewbies/
FAQ: http://kernelnewbies.org/faq/

[Raj]

The Kernel maps the physical address starting from PAGE_OFFSET in
Virtual Address Space that is why we subtract PAGE_OFFSET to get the
physical address from VA. According to my understanding this will be
valid only in Kernel address space and not in user address space.

Again as far as I know, to access the virtual address for a given
physical address, you need to access the rmap structure in the struct
page. This how you can do it,

physical address >> PAGE_SHIFT , this gives you the pfn

node_mem_map + pfn gives the struct page pointer value.

Acess the rmap structure in the page structure. This is a list of all
the virtual addresses where this page is mapped.

Please correct me if I am wrong.

--------------
Raj.

Post by P.F. Tsai
the first 3GB in VA is left for process itself, and the fourth gigabyte is
left for kernel......

--
Kernelnewbies: Help each other learn about the Linux kernel.
Archive: http://mail.nl.linux.org/kernelnewbies/
FAQ: http://kernelnewbies.org/faq/

[Gaurav Dhiman]

Post by Raj
The Kernel maps the physical address starting from PAGE_OFFSET in
Virtual Address Space that is why we subtract PAGE_OFFSET to get the
physical address from VA. According to my understanding this will be
valid only in Kernel address space and not in user address space.
Again as far as I know, to access the virtual address for a given
physical address, you need to access the rmap structure in the struct
page. This how you can do it,
physical address >> PAGE_SHIFT , this gives you the pfn
node_mem_map + pfn gives the struct page pointer value.

can you explain bit more about node_mem_map and mem_map ? What does
these two represent ?

Post by Raj
Acess the rmap structure in the page structure. This is a list of all
the virtual addresses where this page is mapped.

what does rmap represents ?

Please explain bit more, how can we convert the physical address to
the user virtual address ? For converting from virtual to physical, we
need to use process page tables.
I know that page tables are specific to processes, wanted to know is
there any data structure either specific to process ot global, which
can convert the physical address to virtual address of any process ?

Gaurav

Post by Raj
Please correct me if I am wrong.
--------------
Raj.

Post by P.F. Tsai
the first 3GB in VA is left for process itself, and the fourth gigabyte is
left for kernel......

--
Kernelnewbies: Help each other learn about the Linux kernel.
Archive: http://mail.nl.linux.org/kernelnewbies/
FAQ: http://kernelnewbies.org/faq/

--
Kernelnewbies: Help each other learn about the Linux kernel.
Archive: http://mail.nl.linux.org/kernelnewbies/
FAQ: http://kernelnewbies.org/faq/

[Raj]

Post by Gaurav Dhiman

Post by Raj
The Kernel maps the physical address starting from PAGE_OFFSET in
Virtual Address Space that is why we subtract PAGE_OFFSET to get the
physical address from VA. According to my understanding this will be
valid only in Kernel address space and not in user address space.
Again as far as I know, to access the virtual address for a given
physical address, you need to access the rmap structure in the struct
page. This how you can do it,
physical address >> PAGE_SHIFT , this gives you the pfn
node_mem_map + pfn gives the struct page pointer value.

can you explain bit more about node_mem_map and mem_map ? What does
these two represent ?

The memory architecture in LINUX mm follows the NUMA architecture and
hence is organized into nodes. Each node is represented by a pg_data_t
structure. For a NUMA architecture there is a list of pg_data_t called
pgdat_list.
For UMA there is only one pg_data_t structure maintained in a variable
called contig_page_data.

Each node contains a number of blocks of memory called Zones. Each
node also contains an array of struct page called node_mem_map.

typedef struct pglist_data {
struct zone node_zones[MAX_NR_ZONES];
struct zonelist node_zonelists[GFP_ZONETYPES];
int nr_zones;
struct page *node_mem_map;
struct bootmem_data *bdata;
unsigned long node_start_pfn;
unsigned long node_present_pages; /* total number of physical pages */
unsigned long node_spanned_pages; /* total size of physical page
range, including holes */
int node_id;
struct pglist_data *pgdat_next;
wait_queue_head_t kswapd_wait;
struct task_struct *kswapd;
} pg_data_t;

The node_mem_map contains the first page address that the node starts
from. For UMA I think it is the first page after 8MB ( I am not sure
of this though) .

Thus node_mem_map is just an array of struct page of the nodes pages.
The mem_map is the global memory map array of struct page. The mem_map
is used to convert from pfn to struct page* .
For UMA architecture,
mem_map + (physical address >> PAGE_SHIFT) gives the struct page*
which is the physical memory's representation in the Kernel.

Post by Gaurav Dhiman

Post by Raj
Acess the rmap structure in the page structure. This is a list of all
the virtual addresses where this page is mapped.

what does rmap represents ?

The Page tables represent the mapping from the virtual address to the
phsical address. The rmap is a reverse mapping from physical address
to virtual address.

Post by Gaurav Dhiman
Please explain bit more, how can we convert the physical address to
the user virtual address ? For converting from virtual to physical, we
need to use process page tables.
I know that page tables are specific to processes, wanted to know is
there any data structure either specific to process ot global, which
can convert the physical address to virtual address of any process ?

I am sorry. Even I am reading through the code so that is all the
information I have for now. I think the Mel Gorman's Understanding the
LINUX VM Manager might be of more help to you.

Post by Gaurav Dhiman
Gaurav

------------
Raj.

--
Kernelnewbies: Help each other learn about the Linux kernel.
Archive: http://mail.nl.linux.org/kernelnewbies/
FAQ: http://kernelnewbies.org/faq/

[P.F. Tsai]

hi Raj,

I am now getting a little bit more sense about it.

here is my basic thoughts:
for RAM < 896 MB
physical address 0 <==> virtual address 0xc0000000 and thats what
virt_to_phys & phys_to_virt do ( so do __pa and __va). Every physical page
frame is mapped by a struct page that records the page frame's information
and you can find the page via mem_map+pfn, ie pfn_to_page.

for 896 MB < RAM < 4096MB
the *virtual field will record the corresponding virtual address if the
given mapped physical address is larger than 896MB.
does that the way kernel work ? correct me plz if I am wrong.
(will discuss more details after thes basics)

btw, I cant find any function or macro called rmap, is that kunmap you mean
? I need to dig more about this topic, its getting instresting.




--
Kernelnewbies: Help each other learn about the Linux kernel.
Archive: http://mail.nl.linux.org/kernelnewbies/
FAQ: http://kernelnewbies.org/faq/

[P.F. Tsai]

hi Raj,

as u said, I can have a pfn by

pfn = pa >> PAGE_SHIFT;

and then I move a page pointer to the address of corresponding node_mem_map
by calling pfn_to_page like following:

page *mypage;
mypage = pfn_to_page(pfn);

but I can't access the "virtual" filed, cuz it is defined in inside a
#if - #endit section (in 2.6.7 kernel)

#if defined(WANT_PAGE_VIRTUAL)
void *virtual; /* Kernel virtual address (NULL if not kmapped, ie.
highmem) */
#endif /* WANT_PAGE_VIRTUAL */

for 2.4.x

#if defined(CONFIG_HIGHMEM) || defined(WANT_PAGE_VIRTUAL)
void *virtual;
#endif /* CONFIG_HIGMEM || WANT_PAGE_VIRTUAL */

does it mean that the virtual-field is activated (and can be asscessed) if
and only if the HIGHMEN configured and starts to allocate memory
dynamically?

Post by Raj
Again as far as I know, to access the virtual address for a given
physical address, you need to access the rmap structure in the struct
page. This how you can do it,
physical address >> PAGE_SHIFT , this gives you the pfn
node_mem_map + pfn gives the struct page pointer value.
Acess the rmap structure in the page structure. This is a list of all
the virtual addresses where this page is mapped.
Please correct me if I am wrong.
--------------
Raj.

--
Kernelnewbies: Help each other learn about the Linux kernel.
Archive: http://mail.nl.linux.org/kernelnewbies/
FAQ: http://kernelnewbies.org/faq/

[Raj]

Hi Tsai,
I am sorry. I didn't know that you were talking about 2.4.x .
What ever I said about rmap (reverse map) is valid only for 2.6.x
Version.

I am not sure about 2.4.x Version. As far as I know the void *virtual
is for HIGMEM support. In 2.4.x I think the only way to find out all
the virtual address mappings for a given physical address is to
traverse the page tables. That was the main reason why the reverse map
feature was introduced in 2.6.

Rgds,
Raj.

Post by P.F. Tsai
hi Raj,
as u said, I can have a pfn by
pfn = pa >> PAGE_SHIFT;
and then I move a page pointer to the address of corresponding node_mem_map
page *mypage;
mypage = pfn_to_page(pfn);
but I can't access the "virtual" filed, cuz it is defined in inside a
#if - #endit section (in 2.6.7 kernel)
#if defined(WANT_PAGE_VIRTUAL)
void *virtual; /* Kernel virtual address (NULL if not kmapped, ie.
highmem) */
#endif /* WANT_PAGE_VIRTUAL */
for 2.4.x
#if defined(CONFIG_HIGHMEM) || defined(WANT_PAGE_VIRTUAL)
void *virtual;
#endif /* CONFIG_HIGMEM || WANT_PAGE_VIRTUAL */
does it mean that the virtual-field is activated (and can be asscessed) if
and only if the HIGHMEN configured and starts to allocate memory
dynamically?

Post by Raj
Again as far as I know, to access the virtual address for a given
physical address, you need to access the rmap structure in the struct
page. This how you can do it,
physical address >> PAGE_SHIFT , this gives you the pfn
node_mem_map + pfn gives the struct page pointer value.
Acess the rmap structure in the page structure. This is a list of all
the virtual addresses where this page is mapped.
Please correct me if I am wrong.
--------------
Raj.

--
Kernelnewbies: Help each other learn about the Linux kernel.
Archive: http://mail.nl.linux.org/kernelnewbies/
FAQ: http://kernelnewbies.org/faq/

--
Kernelnewbies: Help each other learn about the Linux kernel.
Archive: http://mail.nl.linux.org/kernelnewbies/
FAQ: http://kernelnewbies.org/faq/

[Thekkedath Gopakumar-agt026]

Post by Thekkedath Gopakumar-agt026
Linux kernel provides you with functions like virt_to_phys and friends to
convert a kernel mode virtual address to physical address and vice versa.
The functions you are talking are specifically for converting the
physical addresses to kernel virtual address not to user virtual
address space

Infact, that was what I also meant.

Post by Thekkedath Gopakumar-agt026
As I looked into the functions "virt_to_phys" and "phys_to_virt" in
#define __PAGE_OFFSET (0xC0000000UL)
#define PAGE_OFFSET ((unsigned long)__PAGE_OFFSET)
#define __pa(x) ((unsigned long)(x)-PAGE_OFFSET)
#define __va(x) ((void *)((unsigned long)(x)+PAGE_OFFSET))
As per my understanding PAGE_OFFSET is the starting kernel virtual
address, that means kernel virtual address will start from
PAGE_OFFSET.
My question is, why we are subtracting PAGE_OFFSET from a given
address in __pa() macro, is it because physically kernel code is
placed in lower memory. If this is not the reason, can someone please
clarify this ?

In the case of a 3:1 mapping, PAGE_OFFSET is 3 GB (0xc0000000)
and the page tables are set such that , virtual address 0xc0000000
points to physical address 0x0 and so on. I have also seen systems
with 2:2 mapping and there PAGE_OFFSET is 2GB and hence the
pysical address = virt addr - 2GB.



--
Kernelnewbies: Help each other learn about the Linux kernel.
Archive: http://mail.nl.linux.org/kernelnewbies/
FAQ: http://kernelnewbies.org/faq/

[Thekkedath Gopakumar-agt026]

-----Original Message-----

Post by P.F. Tsai
the first 3GB in VA is left for process itself, and the fourth gigabyte is
left for kernel.
during the kernel booting, va from PAGE_OFFSET was DIRECTLY mapped to pa
from 0 and thats why we subtract PAGE_OFFSET. but that was constrained for
RAM size belows 896 MB.
I am instrested in how it works in the case of RAM size larger than 896 MB
(and less than 4GB)? ram window?
could anyone give me a detail about it ?

Memory above 896 MB is considered to be HIGH_MEM. If you have
a machine with more than 896MB and want to use the HIGH_MEM then
you need to call the page allocation function with GFP_HIGHMEM
and then use "kmap" to get a kernel mode virtual address for that
page.

--
Kernelnewbies: Help each other learn about the Linux kernel.
Archive: http://mail.nl.linux.org/kernelnewbies/
FAQ: http://kernelnewbies.org/faq/

[Thekkedath Gopakumar-agt026]

Post by P.F. Tsai
for 896 MB < RAM < 4096MB
the *virtual field will record the corresponding virtual address if the
given mapped physical address is larger than 896MB.
does that the way kernel work ? correct me plz if I am wrong.
(will discuss more details after thes basics)

Atleast in 2.4 version, for a low memory page the page->virtual
(if defined)will hold the corresonding kernel mode virtual address,
for high memory pages, initially this field is NULL and when they get
mapped (say by calling kmap), the corresponding virt address is stored
here.

Post by P.F. Tsai
btw, I cant find any function or macro called rmap, is that kunmap you mean
? I need to dig more about this topic, its getting instresting.

I think rmap is introduced in 2.6 series, may be you can google
"reverse mapping" and find out.




--
Kernelnewbies: Help each other learn about the Linux kernel.
Archive: http://mail.nl.linux.org/kernelnewbies/
FAQ: http://kernelnewbies.org/faq/

--
Kernelnewbies: Help each other learn about the Linux kernel.
Archive: http://mail.nl.linux.org/kernelnewbies/
FAQ: http://kernelnewbies.org/faq/