- Published on
Create Turbocharged Storage Using tmpfs
- Authors
- Name
- Kevin van Zonneveld
- @kvz
Everyone knows that RAM is so much faster than a hard disk. To illustrate, while a current SATA disk has peak transfer rates of 375 MB/s, current RAM can do a mind blowing 12,500 MB/s! Normally only the system itself makes use of this ultra fast storage, but we can also access this space directly. And that opens a great window of opportunity.
Possible Uses
There is an unlimited number of uses for this technology, but here are 3 from my own experience:
- What if you're running a blog, so successful that your server can't handle it. Although your blogging software caches plain html files in the /cache dir to speed up processing, it still doesn't give you enough performance.
- What if you're an internet host and you want to show off your massive bandwidth by having users download a dummy .bin file of 100MB. You'll find that if many users access this file at the same time, your hard drive becomes the slow factor and you're running into disk IO problems.
- What if you're running a PXE server with an ISO stored on it, and an entire webcluster is accessing this file for installation. Again. Your hard drive will not be able to cope with these kinds of speeds.
If only you could store these files in memory..
How Does It Work?
Everybody who's running a linux server must have seen the /dev/shm on their system.
This not a normal directory on your machine. It is intended to appear as a
mounted file system, but one which uses virtual memory instead of a persistent
storage device. The standard /dev/shm grows automatically as more space is
needed, but is by default limited to half of your physical RAM. If you have
2GB, it can grow to 1GB at most.
So everything you copy to that place is in fact stored in your RAM. And that's cool because your RAM is about 33 times faster than your normal filesystem!
Let's do this
In this article I presume you have basic knowledge of server administration, be careful because you could really mess things up if you've got no clue what you're doing. I warned you!
Use Current Volume
As told before, you probably already have a /dev/shm on your linux system.
So just copy a file to it:
$ cp -af /root/100mb.bin /dev/shm/
And now it's in your RAM! Using this file will be 30 times faster than before!
Enlarge Current Volume
But maybe the limit of half of your physical RAM just does not cut it for you. Then you might want to increase the maximum size of this volume to 4GB:
$ mount -o remount,size=4G /dev/shm
Create a New Volume
Another possibility is to create a brand new memory device. We can do this
with the filesystem type: tmpfs. Let's say you want to create a tmpfs
instance on /var/www/www.mysite.com/ramdrive that can allocate a max of
500MB RAM and that can only be changed by root, but accessed by everyone (like
Apache):
$ mkdir -p /var/www/www.mysite.com/ramdrive
$ mount -t tmpfs -o size=500M,mode=0744 tmpfs /var/www/www.mysite.com/ramdrive
Restore That Volume Everytime Your Server Boots
Easy, just make sure the directory exists, and add the following line to your
/etc/fstab:
tmpfs /var/www/www.mysite.com/ramdrive tmpfs size=500M,mode=0777 0 0
This will create a ramdrive of max 500MB in /var/www/www.mysite.com/ramdrive everytime your server boots. The mode 0777 will give it full access for everybody on your system, so just change that to a suitable umask.
Done.
Pitfalls
Now as with everything too cool, there is a pitfall:
- Everything in tmpfs is temporary in the sense that no files will be created on your hard drive. If you reboot, everything in tmpfs will be lost.
So you will need to create a script that automatically restores the files or applications that you need. You can let this script run at boot time, or schedule it as a cronjob.
Legacy Comments (18)
These comments were imported from the previous blog system (Disqus).
Hi, this is an excelent info Thanks for it!!!
You\'re welcome.
The community has given me so much over the last couple of years I figured it was time to start giving back. But if you like it, please social bookmark this article so it can be of help to others too! Thanks!
How would you use this for gaming? Let\'s say with a linux client of unreal or along those lines.
@ Joseffb: disk IO is not really a bottleneck for gaming is it? Almost every important aspect of a game is loaded in memory anyway sooner or later. The only thing I can think of that you would gain performance on, is load times between scenes or something like that. Not really worth the hassle probably. But purely hypothetically speaking, you could install a game in /usr/local/unreal, after the installation move it to /usr/local/unreal_disk. Then create a tmpfs @ /usr/local/unreal, then copy the contents back like this:
cp -Raf /usr/local/unreal_disk/* /usr/local/unreal/
And then run the game.. And then find it hasn\'t got enough memory left to properly run the game properly ;)
nice read
now do I move, at startup, a frequently used file, say temdb to the ramdrive and use it. I dont mind if I loose it... thet is the Idea... or let it Linux use cahe?.. what is better?
@ Rick: yes it\'s as easy as that.
Cool..
I used this technique to fire my thousands for daily newswires using postfix... gr8 help indeed..
Thanks to Matt as well for his post
http://archives.neohapsis.c...
This is really cool feature of unix ..esp for non-unix guy :-)...and thanks to Kevin to make it so simple
A tmpfs is useful in some circumstances, but not for the three examples you cited. In those cases, once the file is read once it will be cached in memory. There\'s no need for a tmpfs to accomplish this. The cached line in /proc/meminfo tells you how much of a system\'s memory is being used for cache; and you can use the program fincore to see how much of a particular file is cached. It\'s easy to make a demonstration with dd. For example,
With a cold cache:
dd if=sda2.img of=/dev/null
2230345+1 records in
2230345+1 records out
1141936658 bytes (1.1 GB) copied, 28.0823 s, 40.7 MB/s
Now with the hot cache:
dd if=sda2.img of=/dev/null
2230345+1 records in
2230345+1 records out
1141936658 bytes (1.1 GB) copied, 1.75497 s, 651 MB/s
So unless there\'s a lot of disk activity on a system thats causing the files you care about reading to be flushed from cache, don\'t bother with a tmpfs.
@ sciurus: Thanks for the thorough input. I couldn\'t tell you if it\'s because the usual cache is purged to often, not enough files are cached, or maybe it\'s the first read that frustrates performance; but storing cached files on tmpfs greatly improved this sites access times.
@ sciurus
yes its true that most files will be cached but... if say you have a mail server with thousands of small files you can force all the files into ram so that when the file is called for its in ram and ready... cached files must be read first before they can get into ram
so if the file only gets read once and never again (untill its pushed out of cache) the cache was never usefull. a tmpfs mount isnt ment for systemwide use but for when files need fast access times or very high thoughputs
I like to place all my logs in tmpfs (making sure to rotate them hourly as to not fill up the tmpfs mount)
If you do not need to keep your logs or at all... this will save you A LOT of wear and tear on your drives... linux has 3+ month uptimes... so the loss of a bit of logging is nearly zero...
I have 2 raid1 scsi drives have been running since 2001 with no crashs (its 2010) and the drives (hopefully) will last more years.
So, my point is, TMPFS is not just for speed, but can increase the life of your drives... RAM does not burn out, like an SSD drive does.
awsome dude! My site\'s grown and i was desperately in need of a faster way to retrieve cached files.
read ur other post on caching with cache lite, which kead me here.
thanks for both tips.
@ Matt Kukowski: Good point! Many daemons can log to syslog (which can write in bursts) or can write in bursts themselves (like nginx), which is also easier on your disks, and you\'d lose less on power failure/crash than with tmpfs. But yes, if the entries are not that important to you, go for it : )
@ adrian: YW!
is it possible to install a wordpress blog in RAM ?
And how... i have ubuntu server 10.10. Thanks
Thanks a lot for the post, I think that usage of tmpfs might increase my blogs' performance. One, but important question came into my mind.
I've heard, that in comparison to older ramdisk, tmpfs might be swapped by kernel onto harddisk swap partition. Is there any flag, that will force my linux to keep created tmpfs in ram all the time?
@Benoist - It is possible to put wordpress static files in ram (You have to create a script which will sync it with harddrive copy). Wordpress uses database, so You should consider using mysql query caching on another tmpfs storage. I'll do some test & benchmarks soon, and publish results on my blog.
Cheers,
Oh I am trying to configure jack sound server on my PC and one of the prerquites is that tmfs must be turned on.
I tried cat /proc/filesystems but get the error msg bash: cat/proc/filesystems: No such file or directory
Is there a way to chk if tmfs is turned on.
Thanks for this informative writeup.