lotus domino on aix memory usage explained_lotus notes

The IBM AIX platform handles memory differently than all other Lotus Domino supported platforms. AIX uses a segmented architecture that has both advantages and disadvantages. The primary advantage is increased performance. Of the disadvantages, a 32-bit program must be cognizant of the segmented architecture to take full advantage of the 32-bit address space. This article describes the possible segment layouts for a 32-bit application on the AIX platform as well as the layout that Lotus Domino uses currently. It also shows what happens to the layout as you use the different Domino memory options.

This article is intended for experienced Lotus Domino system administrators familiar with the AIX platform.

Understanding segment layouts

By default, the 32-bit virtual address space on AIX consists of 16 segments of 256 MB each. For any application that uses the default segment layout, the virtual address space looks like the one shown in figure 1:

• The first segment (0x0) is used for kernal text and data.
• The second segment (0x1) is used for user text.
• The third segment (0x2) is used for user stack and data.
• Segments four through thirteen (0x3 to 0xC) are available for user process if either shmat() or mmap() is called.
• Segment fourteen (0xD) is reserved for shared library text.
• Segment fifteen (0xE) is available for user process.
• The last segment (0xF) is used for pre-process shared library data.

For anyone unfamiliar with the terms used in figure 1, here are definitions:

• Text. Code that is read-only and executable. It can be of three-type kernel code, user code, and shared library code.
• Data. Read/write data area that can be of three type kernel data, user data, and shared library data.

NOTE: Shmat() and mmap() are used in Lotus Domino to get shared memory.

Figure 1. Virtual address space example

Programs that use the AIX large memory model have the following memory layout shown in figure 2:

• The first segment (0x0) is used for kernal text and data.
• The second segment (0x1) is used for user text.
• The third segment (0x2) is used for user stack and data.
• Segments four through seven (0x3 to 0x6) are reserved by the process heap.
• Segments eight through thirteen (0x7 to 0xC) are available for user process if either shmat() or mmap() is called.
• Segment fourteen (0xD) is reserved for shared library text.
• Segment fifteen (0xE) is available for user process.
• The last segment (0xF) is used for pre-process shared library data.

Current versions of Lotus Domino use the AIX large memory model.

Figure 2. AIX large memory model

In AIX 5L version 5.2 and later, there is a very large memory model. This is the model that we hope to use in the future with Lotus Domino, but it is not currently used. This model has three forms: the first form is for programs that have a process heap (user data) less than 2.5 GB and greater than 256 MB (see figure 3). The model looks similar to the default AIX segment layout, but segments four through eight (0x3 to 0x7) are available for user heap if dynamic segment allocation (DSA) is used. Also, segments 0xA to 0xE are available for user process if shmat() or mmap is called() by dynamic segment allocation.

Figure 3. AIX very large memory model: first form

The second form is for programs that have a process heap greater than 2.5 GB as shown in figure 4 in which:

• Segment one (0x0) is reserved for kernal text and data.
• Segment two (0x1) is reserved for user text.
• Segment three (0x2) is used for the user stack. However, shared library text and data are also used in this segment (0x2).
• Segments four through sixteen (0x3 to 0xF) are available for the user heap if used by dynamic segment allocation.
• Segments 0xB to 0xF are available for the user process if shmat() or mmap() is called by dynamic segment allocation.

Figure 4. AIX very large memory model: second form

The third form is for programs that have a process heap less than 256 MB (see figure 5). In this form:

• The first segment (0x0) is used for kernal text and data.
• The second segment (0x1) is used for user text.
• The third segment (0x2) is used for user stack and data. However, shared library text and data may also be stored in this segment.
• The rest of the segments (0x3 to 0xF) are available for the user process is shmat() or mmap() is called by dynamic segment allocation.

Figure 5. AIX very large memory model: third form

Back to top

Lotus Domino memory control options

Now that you have been exposed to some basic information about AIX memory models, lets explore the model that Lotus Domino uses and how the Domino memory control options change the virtual address space.

The rest of this article shows you how to change the segment layouts by configuring settings in the Domino servers Notes.ini file. Before you proceed, here is a legend to help you understand the tables of information better.

• Program name. This is an example of a typical Domino application.
• Kernal text and data. This memory is shared.
• User text. This memory is shared for all programs with the same executable file (for example, three update tasks use only one chunk of memory shared between the three tasks.)
• User stack. This segment is unique to each process.
• Process heap. This segment is unique to each process.
• Shared memory. This segment is shared by all Domino processes.
• Shared library text. This segment is shared by all processes on the system.
• Unavailable segment. This segment is currently unusable by Lotus Domino.
• Shared library data. This segment is unique to each process.
• Non-Domino. This segment is shared and used by third-party applications.

Program shared data shared by all Domino applications starts at 256 MB and grows by 256 MB segments as needed until the maximum number of free segments is reached.

This first table shows the results of having no Domino memory settings enabled.

Program name	Kernal text/data 256 MB	User text 256 MB	User stack 256 MB	Process heap 512 MB	Shared memory 2 GB	Shared lib text	Unavailable segment	Shared lib data
Server	1	2	3	4 and 5	6 to 13	14	15	16
HTTP	1	2	3	4 and 5	6 to 13	14	15	16
Update	1	2	3	4 and 5	6 to 13	14	15	16
N/A	1	2	3	4 and 5	6 to 13	14	15	16
Process stack		Process private data		Domino shared memory		Third-party API application shared memory
256 MB		512 MB		2048 MB		0 MB

This is the layout of the shared memory:

• NSF Buffer Pool (UBM=3/8 shm) = 750 MB
• Other Domino shared memory = 1250 MB

Configuring the ConstrainedSHMSizeMB setting

The ConstrainedSHMSizeMB setting in the Domino servers Notes.ini file restricts shared memory. There are two ways in which to use this setting. If ConstrainedSHMSizeMB=1, then shared memory is restricted to a default size. In AIX, the default size is 2.25 GB. You can also specify a size in megabytes for this setting, which is what we have done in this article for demonstration. For example, if you change the setting as follows:

ConstrainedSHMSizeMB = 1744

you may find the following results listed in this table.

Program name	Kernal text/data 256 MB	User text 256 MB	User stack 256 MB	Process heap 512 MB	Shared memory 1.744 GB	Non-Domino 256 MB	Shared lib text	Unavailable segment	Shared lib data
Server	1	2	3	4 and 5	6 to 12	13	14	15	16
HTTP	1	2	3	4 and 5	6 to 12	13	14	15	16
Update	1	2	3	4 and 5	6 to 12	13	14	15	16
N/A	1	2	3	4 and 5	6 to 12	13	14	15	16
Process stack		Process private data			Domino shared memory		Third-party API application shared memory
256 MB		512 MB			1744 MB		256 MB

This is the layout of the shared memory:

• NSF Buffer Pool (UBM=3/8 shm) = 654 MB
• Other Domino shared memory = 1099 MB

If you configure ConstrainedSHMSizeMB= 1744 in the Notes.ini file and set the dataseg used to 3 (the number of data segments), you may find the following results in this table.

Program name	Kernal text/data 256 MB	User text 256 MB	User stack 256 MB	Process heap 768 MB	Non-Domino 1.744 GB	Shared lib text	Unavailable segment	Shared lib data
Server	1	2	3	4 to 6	7 to 13	14	15	16
HTTP	1	2	3	4 to 6	7 to 13	14	15	16
Update	1	2	3	4 to 6	7 to 13	14	15	16
N/A	1	2	3	4 to 6	7 to 13	14	15	16
Process stack		Process private data		Domino shared memory		Third-party API application shared memory
256 MB		768 MB		1744 MB		0 MB

This is the layout of the shared memory:

• NSF Buffer Pool (UBM=3/8 shm) = 654 MB
• Other Domino shared memory = 1099 MB

In the following table, ConstrainedSHMSizeMB is set to 1024 MB.

Program name	Kernal text/data 256 MB	User text 256 MB	User stack 256 MB	Process heap 512 MB	Shared memory 1024 MB	Non-Domino 1024 MB	Shared lib text	Unavailable segment	Shared lib data
Server	1	2	3	4 and 5	6 to 9	10 to 13	14	15	16
HTTP	1	2	3	4 and 5	6 to 9	10 to 13	14	15	16
Update	1	2	3	4 and 5	6 to 9	10 to 13	14	15	16
N/A	1	2	3	4 and 5	6 to 9	10 to 13	14	15	16
Process stack		Process private data			Domino shared memory		Third-party API application shared memory
256 MB		512 MB			1024 MB		1024 MB

This is the layout of the shared memory:

• NSF Buffer Pool (UBM=3/8 shm) = 384 MB
• Other Domino shared memory = 640 MB

Configuring the PercentAvailSysResources setting

The PercentAvailSysResources setting in the Domino servers Notes.ini file lets you control memory allocation on your server. Values for this setting range from 2 percent to 100 percent. In the following table, PercentAvailSysResources is set to 25 percent on a system with 4 GB of memory. In other words, 1 GB of memory has been allotted to the Domino server.

Program name	Kernal text/data 256 MB	User text 256 MB	User stack 256 MB	Process heap 512 MB	Shared memory 2048 MB	Shared lib text	Unavailable segment	Shared lib data
Server	1	2	3	4 and 5	6 to 13	14	15	16
HTTP	1	2	3	4 and 5	6 to 13	14	15	16
Update	1	2	3	4 and 5	6 to 13	14	15	16
N/A	1	2	3	4 and 5	6 to 13	14	15	16
Process stack		Process private data		Domino shared memory		Third-party API application shared memory
256 MB		512 MB		2028 MB		0 MB

This is the layout of the shared memory:

• NSF Buffer Pool (UBM=3/8 shm) = 384 MB
• Other Domino shared memory = 1664 MB

In this configuration, the setting of the NSF Buffer Pool used 3/8 * (RAM * .25). There is no defined limit to the growth of Domino shared memory other than the operating system imposed limit of segments.

The next table shows the results of setting NSF_BUFFER_POOL_SIZE_MB= 512 and PercentAvailSysResources=25 in the Notes.ini file.

Program name	Kernal text/data 256 MB	User text 256 MB	User stack 256 MB	Process heap 512 MB	Shared memory 2048 MB	Shared lib text	Unavailable segment	Shared lib data
Server	1	2	3	4 and 5	6 to 13	14	15	16
HTTP	1	2	3	4 and 5	6 to 13	14	15	16
Update	1	2	3	4 and 5	6 to 13	14	15	16
N/A	1	2	3	4 and 5	6 to 13	14	15	16
Process stack		Process private data		Domino shared memory		Third-party API application shared memory
256 MB		512 MB		2028 MB		0 MB

This is the layout of the shared memory:

• NSF Buffer Pool (UBM=3/8 shm) = 512 MB
• Other Domino shared memory = 1536 MB

In this configuration, the setting of the NSF Buffer Pool is directly configured, and the PercentAvailSysResources is ignored. There is no defined limit to the growth of Domino shared memory other than the operating system imposed limit of segments.

Recommendation

We recommend that you do not use any form of ConstrainedSHMSizeMB. In the currently used AIX large memory model, ConstrainedSHMSizeMB should be used only to support third-party applications that require shared memory. If you use it, then Lotus Domino has less virtual address space to use. This is unlike all other Domino supported platforms. On other platforms, using ConstrainedSHMSizeMB to reduce the shared memory automatically increases the amount of private data space for the application.

To illustrate, heres an example: The Domino server is running a very heavy Lotus Domino Web Access HTTP load that requires 1024 MB of private data space. On AIX, you use dataseg –f 4 * in the Domino program directory to enable all Domino program files to use four segments of private memory. On other platforms, you need to use the operating system imposed virtual address space size. Then you subtract the space used for program text and shared library text. This gives you the amount of virtual data space available to Lotus Domino. In this example, you subtract the 1024 MB from the virtual data space available to Lotus Domino and use this for the ConstrainedSHMSizeMB setting.