David e-mailed my dad (I’m just Tim) with a question about using RSYNC in Windows to backup to FreeNAS when the data is in two different locations. Since my dad doesn’t know anything about FreeNAS, I figured I’d answer:

Dear Mr. Fehlman,

Congratulations on your excellent series of articles on FreeNAS. I have got a Dell PIII with 640MB RAM and a single 30GB disc up and running with FreeNAS; the disc has two partitions one for FreeNAS and the second for data. Ultimately, what I would really like to aim for is a RAID FreeNAS to backup family photos, and MYOB business data….and here lies a problem.

It would appear that in FreeNAS each partition represents a single share point. With RSYNC, we need a matching share on the Windows box, and this Windows share (represented by a virtual directory in DeltaCopy) can only point to a single folder. Am I correct in stating that if I wanted to backup two different folders on the Windows box (say, MYPHOTOS and MYOBDATA), I would need two virtual directories in DeltaCopy, and thus would need two matching data partitions on the FreeNAS box? I note your link to “How to partition hard-drives for FreeNAS“, but I’m starting to get out of my depth by this stage (I consider myself a newbie luddite).

Many thanks for your comments,

David

Well, David, you are correct. There needs to be a 1:1 ratio between folder and backup on the FreeNAS using RSYNC. So, how do you go about backing up several different folders that are all over your hard drive? There are three different ways that this can be accomplished.

1. Move All Data One Folder

The first thing that comes to mind is that you can store all of your data in subfolders under one folder (e.g. My Data). Then, all you would have to do is set up one partition on the FreeNAS box and RSYNC the one folder to the FreeNAS partition.

2. Create an Intermediate Backup Folder

“But, what if you can’t move the data?”, you ask, “There are a lot of programs out there that do not let you specify where the data can be stored. How would you back up something like that?”

You could create an intermediate folder where you use a program such as Cobian Backup to copy all of the files to one folder and then RSYNC that folder to the FreeNAS box. Unfortunately, you now have two copies of your data residing on your hard drive which has a tendency to get pretty huge after a period of time.

There is also the issue of the initial file backup still running when RSYNC fires up. This means that you will be copying your data to the FreeNAS box prior to the backup completing. Definitely an issue.

3. Use NTFS Junction Points

The solution that I thing is the geekiest (thus, the coolest) is to use NTFS junction points. Essentially, you make the system think that an empty folder in a backup folder actually contains your data. Then, when RSYNC goes to copy the data, it will follow the junction points and backup everything you want.

Junction Link Magic
If you are new to NTFS junction points, this can be a bit of an odd concept to get used to. The way I wrap my head around it is to think of them like “stealth” shortcuts. Just like a shortcut on your desktop can point to a file or folder somewhere else on your hard drive, junction points point to other folders on your hard drive. The difference between shortcuts and junction points is that the file system sees shortcuts as files with an .LNK extension but it sees junction points as just another folder on your hard drive.

If you are interested in working with junction points, there are a lot of different programs out there that will allow you to create and delete them on your system. The one that I use the most is called Junction Link Magic and is available as freeware (of course). There are a lot of others out there but this is just the one that I am familiar with.

Junction Point Loop
Just a quick word of warning. You can really mess up your system if you go deleting junction points without understanding what they actually are doing. I suggest that you only create new junction points in folders that you have just created for the purpose of redirecting information. I also suggest that you do not delete any junction point that you dis not create. This is another great way to destroy your system. And finally, do not create a junction point that points to a folder higher up in the directory tree. This will create a junction point loop and if you are backing up this folder, you will never finish your backup.
Also be aware that you can only perform NTFS junction points on Windows 2000 and above. Your hard drive also needs to be formatted NTFS.

RSYNCed Data Unavailable

Data UnavailableNow, this brings up another problem. People are finding that they are not able to access their RSYNCed data. This is a know issue but there is a solution….which I will tell you about tomorrow!

Similar Posts:

If you found this post useful, why don't you buy me a cup of coffee to show your gratitude?

TrophyCongratulations to the FreeNAS team on being named the Project of the Month on SourceForge! This is a huge honor but it is definitely well deserved. Thanks for all your hard work.

SourceForge interviewed the development team and it is a very interesting read. I have included a couple of tidbits. When asked what was on their project wish list, this is what they had to say:

Olivier: A solution to the Samba performance problem under *BSD A powerful and stable software RAID 5 solution under FreeBSD.

Dan: Quota management

Volker: Improvement of the mount and share workflow

Support for plugins

Filesystem encryption

Bob: Full user and partition controls

Power user plugins (the ability for a power admin to execute specific commands on startup)

They also laid out the milestones that they have for the project (remember, this is still a beta project!):

  • 0.69: Add lots of FreeBSD geom class (gconcat, gstripe, geli, etc.)
  • 0.7: Disk/share management will be completely rewritten to add user permissions and quotas
  • 0.8: Add moniroting features: SNMP, syslog, alert, etc.
  • 0.9: Work on stability, security, and bugs fixes.
  • 1.0: The release!

Now, FreeNAS is in some pretty good company. The SourceForge Project of the Month list reads like a who’s who of the open source industry. Here is the list going back to 2002. I’m sure you are going to see some familiar faces in the list.

2007

2006

2005

2004

2003

2002

Once again, congratulations on a job well done!

Similar Posts:

If you found this post useful, why don't you buy me a cup of coffee to show your gratitude?

CD DetailsApparently, I’m not the only one who is working hard these days. FreeNAS has released yet another beta version (FreeNAS-0.683b). This is the second beta in less than a week!

From the website:

Notes:
Major bug fix:

  • Fix the ‘disk is mounted’ bug when formatting disk.
  • Upgrade smartmontools, permit to mount CD/DVD drive and minor bug fix and improvement.

Changes:

  • Upgrade Smartmontools from 5.36 to 5.37
  • Add the HPN-SSH client (for preparing rsync and unison over ssh)
  • Permit to mount CDROM drive
  • fixed the display of ‘DMA’ in the place of the size when controller found non-ATA66 cable
  • WebGUI: Add all RAID information on the Diagnostic information page, More easy fsck page to use, Improve graid5 tools page
  • New language: Romana (thanks to Laurentiu Florin Bubuianu)
  • Permit to configure the hosts file
  • Fix the disk check mounted bug when formatting a disk.
  • Add gettext support on PHP (prepare migrating languague file to .po)

I think the one big addition to this beta is the ability to mount CDs and DVDs. Up until now, I have always removed the DVD drive when I was finished to the base installation as I wanted to use the IDE connection for a hard drive. But, it might be useful to have a USB drive around so that you can quickly plug it into the FreeNAS box and then share a DVD to the network.

Similar Posts:

If you found this post useful, why don't you buy me a cup of coffee to show your gratitude?

31533027_e69f4a3878_m.jpgMy favorite open source NAS software, FreeNAS, has released an updated beta version of the software.

Here are the release notes:

  • Bug fixes from the 0.681: Install from CDROM, LAN page modification that crash the WebGUI, fsck that don’t umount disk.
  • New languages added and SSH replaced with HPN-SSH.
  • Looking for testers about the new HPN-SSH implementation!

and this is a list of the changes that have been implemented in this release:

  • Fixed 0.681b bug: missing function (fdisk) for installing ,fsck tools that don’t umount and LAN page that crash webGUI.
  • CIFS: hide dot files for prevent Netatalk system directory to be delete
  • CIFS: add two parameters for trying to improve performance (samba perf suck under FreeBSD…)
  • SSH: Add HPN-SSH patch by using openssh-portable FreeBSD port
  • Fixed fstab warning message while booting (Thanks to Graham Wharton)
  • Modified build scripts
  • Hiroyuki Seino patch : CP932 charset added, and webgui bug fix
  • New language: Traditional and simplified Chinese (Thanks to Baikuan Hsu), Dutch (thanks to Kris Verhoeven)
  • Add warning when choosing install with 2 partitions (Thanks to Bob)

You can download the latest beta version from the FreeNAS SourceForge website.

Similar Posts:

If you found this post useful, why don't you buy me a cup of coffee to show your gratitude?

agentread.pngI remember the first hard drive that I owned. It has a whopping 20 MB of disk space. I recall a conversation I had with someone about a new 500 MB drive and wondering what in the world you would do with all that storage capacity. Apparently, we figured that one out!

So, I am really excited that Hitachi has created the world’s first 1 TB hard drive! The Deskstar 7K1000 will sell for about US$400 and will allow you to store up to 3,000 hours of video or about 200,000 songs in MP3 format. It will be available sometime in Q1 of this year.

According to the website, 1 TB drives are going to become a necessity as everything starts to go digital:

Consumers have entered an era where everything they capture on digital still or video, listen to and write, can be saved and shared for future enjoyment. In the connected home, storage becomes the epicenter of the home network, linking the living room, the office and the bedroom together in a truly digitally-connected home. The family’s games, home movies, photos and music can be stored and accessed from a variety of digital entertainment devices within the home. As the distribution of video content to social networking sites and to handheld devices becomes more commonplace, storage capacity requirements will reach new heights. In this scenario, consumers can begin to imagine what they would do with all that capacity. With 1TB, Hitachi’s Deskstar 7K1000, for PCs and home media centers and the CinemaStar 7K1000 for DVRs help to usher in this new era.

Here are the specs:

  • 1000/750 GB – SATA (GB = 1 billion bytes, accessible capacity may be less)
  • 148 billion bits per square inch maximum areal density
  • 1070 Mb/s max. media data rate
  • 8.7 ms average seek time (with command overhead)
  • 7,200 RPM, 4.17 ms average latency
  • Serial-ATA 3.0Gb/s
  • 32 MB data buffer – SATA
  • 26.1 mm in height (max)
  • 700g in weight (max)
  • 5/4 platters, 10/8 recording heads – SATA
  • 300 G/1 ms pulse non-operating shock
  • 9.0 (5 disk)/8.1 (4 disk) watt idle power – SATA
  • 2.9 Bels typical idle acoustics
  • 5-60 degrees C operating temperature

I imagine that if I were to drop a couple of these drives into a FreeNAS box, it would make a pretty sweet storage unit!

P.S. If you would like to see some edutainment, check out License to Read.

Similar Posts:

If you found this post useful, why don't you buy me a cup of coffee to show your gratitude?

« Previous PageNext Page »