How to use SSD as RAM?

I will immediately make a reservation that I do not need a ram-disk, but I need to make sure that the SSD is considered by the system as a certain bar(s) of RAM memory. The question about speed and latency is not raised.

That is, I define a certain file or the entire disk on the SSD, which is considered as a memory slot in the system.

Virtual (swap) memory is not suitable because it is "transparent" or invisible to the software.

For example: there is 4GB of real RAM and 128GB SSD. I would like to get a part of it SSD as RAM for VMs and in general for programs. With the desired scheme, I can provide up to the maximum for the processor (64GB), rather than the maximum available RAM (4GB)

I will consider all options, except for the option "buy RAM and do not swap brains".

Thank you all for explaining the theory of RAM and page organization of memory.

We are talking about a purely software method of" faking " memory, so that you can allocate more memory for programs than there is on board. The example with a VM is clear: none of the VMs it will not allow you to allocate more memory than there is RAM, even if you have a swap file 16 times larger.

Author: Adokenai, 2020-01-08
Source

1 answers

You need to use page file or swap. I will try to explain in detail why.

In the beginning - what kind of memory is there at all.

Actually, for RAM, there are physically two interfaces that support RAM, one DDR (RAM) razem, the second PCI (or PCI-express) as a living example - the memory of the video card. There are other outdated AGP PCMMCI interfaces, I can't list them all, but they are already in the past.

If your device is not physically inserted into the first interface not in the second case, your device can not physically be a direct source of RAM because there is no other protocol of exchange between the CPU and the device than the above. Therefore, it is hypothetically possible to "solder" the adapter. Truth... how to make PCI memory available to a PC is a separate question that I can't answer unequivocally.

The SATA interface and IDE do not support direct RAM exchange, so its devices cannot be sources RAM. UPD after reading the Internet, I found PCIe SSD disks, they can not be used as RAM because the PCIe-SSD disk is embedded with the SATA program, and not the RAM program. Whether it is possible to sew another program that PCIe would be seen as RAM (very slow) memory is questionable. Theoretically, you can argue, in practice, I have not met, and killing an SSD with flashing is not fun.

So physical RAM is available through the memory controller between PCI and DDR-RAM all.

SATA IDE USB devices support DMA, but DMA is an incomplete access to memory, and access to memory through an "intermediary", i.e. you must first set the address counter to the DMA register, then tell the device that it can use DMA, and then the device can use DMA and can not use it-there is a complex exchange method that is hardware radically different from operational access to memory. knows how much memory is on the device, although it simplifies the work.

Programs for Windows and Linux usually work with virtual memory, which means that the memory can be RAM, can be disk (and SSD, too) or it may not be at all. The program memory is divided into blocks (at least 4096 bytes are larger), and RAM contains only those blocks that the program is currently using. The remaining blocks can be as follows on the disk, or not at all. When trying to access virtual memory which is non-operational under certain conditions, the memory is replaced by the operational one, i.e. The OS looks at the disk memory or not. If the memory is disk-the OS searches for a piece of RAM, reads from disk to RAM a chunk, and the memory goes back from disk to RAM. If there is no free operational piece, the OS can reset a piece of little - used memory. (calculated) memory to disk, so the OS will always find a free chunk.

This mechanism is called swap, page file, swap file. I.e. the amount of virtual memory is the amount of physical RAM + the amount of the swap file.

Your proposed algorithm is very similar to the algorithm described above, but with the difference that you do not want to tell programs anything about the availability of disk memory. I do not think that this is good, because if the" core " of the OS gets to the disk, then everything will be terrible slow down. In the OS, there is a distinction between which areas can be made disk memory, and which can not. Theoretically, such a program can be created. Unfortunately, no one will help you do it, and it will take you years to implement it, and I hope you understand that there will not be much benefit from such a program.

For this purpose, a swap file is usually used.

Question We are talking about a purely programmatic way of" faking " memory, so that you can allocate more memory for programs than available on board The answer is - it is possible, but not as usual. It is very desirable to allocate memory through VirtualAlloc, because the regular memory manager will be in a stupor, it is most likely not designed for such a volume. It is necessary to allocate a multiple of the size of the memory page (to find out the size, you need to call GetSystemInfo and take the dwAllocationGranularity parameter). And then the OS itself will distribute where you will have RAM, and where disk. We must understand, that in x86 this can not be done within a single process (because theoretically the maximum amount of virtual memory is 4Gb, and somewhere half of them is reserved for the OS and libraries, the linear free piece is even less than half), for x86 programs you will have to create more than one process, and for each process you can allocate somewhere 1Gb (conditionally). The memory can be read using ReadProcessMemory for example. Some web browsers create a lot of processes, I think for this goal. You can also use the AWE window (but in practice only through the admin rights or the service).

For x64 it is possible, perhaps with one feature. I think that it is impossible to allocate a huge volume in one piece. But you can allocate 64 times 1Gb (or more), and it is through VirtualAlloc. Under Linux, I will not say exactly how, but I think similarly. Try it, there will be more questions-ask.

Question none of the VMs will allow you to allocate more memory than there is RAM - The answer also allows how. Maybe not all programs, but some programs can definitely "eat" more than RAM.

P.S. Don't forget that the swap file doesn't have to be a file. Under Linux, you can allocate a partition in the partition for paging, and then a certain part of the disk will be used for the paging file, and due to the lack of a file system on this part, the work will be faster.

P.S. UPD more. The speed of DDR4 is 25600 MB / s, DDR3 is from 6400 MB / s to 12800. The PCIe speed is max 64000Mb/s, but the SSD speed - I found a maximum of 3200 Mb / s estimated, 2200 Mb/s for sequential reads (actually it will be less) Intel SSD 750 http://fcenter.ru/online/hardarticles/hdd/38592-SSD_s_interfejsom_PCI_Express_obzor_i_testirovanie_pyati_modelej I understand one of the most expensive and best... and given the element base, it is unlikely that the speed will be comparable to the speed of RAM. In practice, DDR (RAM) will be easier and cheaper to manufacture.

UDP: RAM disk. There are RAM disks, you can write information to them, but the information from them is lost after the power is turned off. For example, Gigabyte i-RAM (GC-RAMDISK) in PCIe is wide open. They are probably well suited for paging files or temp folders. RAM is inserted there, but it is used as a disk, i.e. on the contrary than in the question-memory is used as a disk.

I will also note that PCIe v 1.0 can be done on a microcontroller, v2. 0 can be conditionally done on a microcontroller.

About PCIe. Speeds of 2.5 GT/s to 10.0 GT/s(8 lines) for PCIe1. 0 are available at 250MHz. For 99% of logic and microcontrollers, this is the ceiling (provided that the logic parallelizes the signal, the units of MS will pull 250 MHz (D-triggers are at 350 MHz for TI)). To implement higher speeds, you need to use expensive FPGAs, which also need to be able to program and solder. Izvesny FPGAs for which there is documentation - no more than 10-20 pieces (not taking into account the division into series), plus something else is written in the documentation about PCIe licenses. Therefore, it will not be possible to "touch" PCIe 2.0+ itself, and PCIe will be developed exclusively by companies that can afford to make chips themselves, or at least can program FPGAs, i.e. new PCIe products from ordinary white people can not be expected, the road is still closed to us there. Unless some individual enthusiasts can master the FPGA. Adapters or PCI-phy-layer chips for PCIE for some reason still do not exist and probably will not be because this area is probably not in demand.

References

 12
Author: nick_n_a, 2021-01-26 16:43:15