The Best of Linux.com Weekend Project from 2011

Weekends are for relaxing, spending time with friends… and tackling those tech projects that you never have time to get to during the week. The weekend project is one of the most popular features here on Linux.com, and we had a bumper crop of excellent projects in 2011. Here’s 10 of the best from 2011, which include everything from better ways to upgrade your system, to getting a leg up on Web projects.

Reproducing your current environment on when you upgrade isn’t always easy. Nathan Willis undertakes a from-scratch reinstall, and provides valuable lessons that you can use when you tackle your next migration.

Debian developer Joey Hess starting writing etckeeper after unsatisfying experiments with other people’s attempts to shoehorn /etc/ into a Git repository. A few people had done so successfully, but ran into two major problems: what to do when a package installation made changes to the directory or a file (i.e., and the user could not enter the usual log entry), and what to do about metadata changes like file permissions. This weekend project shows how you can control your configuration with etckeeper.

Ever had a Web project in mind, but got stalled at the prospect of having to worry about the site design? If code, and not design, is your strong point you’ll want to take a look at Twitter’s Bootstrap.

For the security conscious, there is always room for another weapon against attackers. Firewalls, intrusion detection systems, packet sniffers – all are important pieces of the puzzle. So too is Honeyd, the “honeypot daemon.” Honeyd simulates the existence of an array of server and client machines on your network, including typical traffic between them. In this weekend project, you’ll learn how to use HoneyD on Linux to fool attackers.

If you’ve ever needed to edit one or more files to make quick changes, you’ve no doubt found that doing it using a text editor can be a slow slogging process. Linux, thankfully, has a number of tools that make it easy to do this non-interactively. One of the best is sed, a “stream editor” that can help you make quick work of filtering and transforming text. Use this weekend project from 2011 to get to know GNU sed.

One of the keys to using GNU sed successfully is knowing how to use its regular expressions. If you look over sed scripts without knowing regular expressions, the effect can be pretty disconcerting. Don’t worry — it’s not as confusing as it looks. If you’ve read the “get to know GNU sed” piece, step up your sed game with an intro to using sed regular expressions.

Just about every program has spell checking, but what if you’re grammatically challenged? Most apps don’t have a grammar checker built-in, even though they should. Want to get a leg up on your writing? Learn how to get grammar checking for your open source office suite. Note that After the Deadline not only works with LibreOffice, it also works with tons of other apps – including my favorite, Vim.

The Gentoo-based SystemRescue CD/USB is one of the very best rescue distros, packing amazing functionality into a 350MB image. It can rescue Linux, Unix, Mac, and Windows systems, and recover data from almost any media. With this weekend project, you’ll learn how to create a SystemRescue live USB stick, and recover data from failing drives.

Ubuntu switched to Thunderbird a while back, and Linux Mint has defaulted to the Moz mailer for quite some time. If you want to make the most of Thunderbird, though, you should check out this weekend project from July on adding conversations and calendaring to Thunderbird.

Have a book you want to publish? Ready to self-publish that Great American Novel that you’ve been tinkering with since high school? These days, the barriers to publishing a book are lower than ever. All you need is a manuscript and an open source application like Sigil to tame the text and massage it into EPUB format. If writing and publishing is on your resolution list for 2012, learn how to write and publish eBooks on Linux with Sigil.

Have some ideas what you’d like to see on the weekend project for 2012? Give us a shout in the comments. Happy New Year!

Comments (0)
You must be logged in to post a comment. Please register if you do not have an account yet.

View the original article here

Weekend Project: Get to Know Btrfs

The Butter/Better/B-tree Filesystem, Btrfs, is supposedly destined to become the default Linux filesystem. What makes it special, and what's wrong with good old tried-and-true Ext2/3/4?

Linux supports a gigantic number of filesystems: removable media, network, cluster, cloud, journaling, virtual machine, compressed, embedded, hardware inter-connect, pseudo-filesystems that live only in memory, Mac and Windows filesystems, and many more.

You are doubtless familiar with the general-purpose Ext2/3/4, JFS, XFS, and Reiser filesystems that we use on our desktop PCs and servers. With all of these filesystems cluttering up the landscape, what is the point of yet another one? (There is even YAFFS: Yet Another Flash File System.)

The point is meeting new needs and workloads, and building functionality into the filesystem rather than relying on a herd of external utilities. Btrfs is rather like a blend of features from ReiserFS and ZFS, Sun's advanced copy-on-write/volume manager/RAID/snapshot/etc. filesystem.

Many Linux users yearn for a native port of ZFS, but its GPL-incompatible license (the Sun CDDL) ensures that Sun's implementation (now Oracle's) can't be included in the Linux kernel.

Even so, you can't keep a good hacker down, and so there are two ports for Linux. One is ZFS on FUSE, which runs ZFS in user-space. It's included in a lot of distros so it's an easy installation. The other one is ZFS on Linux. This is a build of ZFS as a kernel module for users to install, and so you get kernel support without a GPL violation because it is not distributed with the kernel.

It's great having those options to try out ZFS, and I applaud the maintainers of these ZFS projects. Still, it looks like Btrfs is going to take the place that ZFS could have owned were it not for its incompatible license. Oracle is the primary sponsor of Btrfs, and plans to make it the default filesystem in Oracle Unbreakable Linux sometime in 2012. Btrfs isn't just an Oracle project, but has a lot of community support from the Linux kernel team and many Linux distributions. Odds are it's included in your favorite distro. (Run cat /proc/filesystems to see what filesystems your Linux supports.)

So what does this amazing super-duper filesystem do? How about a handy bullet-pointed list to answer this question?

RAID 0, 1, 10COWIncremental backupOnline defraggzip and LZO compressionSpace-efficient packing of small filesDynamic inode allocationChecksums on data and metadataShrink and grow storage volumesExtentsSnapshots16 EiB maximum file size

Planned features include RAID 5 and 6, deduplication, and a ready-for-primetime filesystem checker, btrfsck. You can try out btrfsck now because it is included in btrfsprogs. (Which of course Debian/Ubuntu/Mint etc. changes to btrfs-tools, and Fedora calls it btrfs-progs.) But it is not ready for production systems yet.

Putting the finishing touches on btrfsck is the last big step before Oracle makes it the default filesystem in their next Unbreakable Linux release. Fedora 16 Linux was supposed to default to Btrfs, but now they're aiming for Fedora 17 in May 2012.

I'm a big fan of RAID 10, which is RAID 1+0, mirroring and striping. It is expensive of disks because only 50% of your total disk capacity goes to storage. But it is simple, robust, and fast. Half your disks can fail without losing your data. I got burned out on RAID 5 and 6 years ago; perhaps I had bad RAID mojo, but I experienced a lot of failures, and they are slow. It seemed the systems under my care were more adept at propagating parity errors than operating correctly. So for me, RAID 5 and 6 can sit on the back burner indefinitely as long as I have RAID 10.

16 EiB is exbibytes, a measurement close to the more commonly used exabyte. An exbibyte is 1,024 pebibytes. In comparison Ext4 maxes out at volumes with a maximum size of one exbibyte and file sizes up 16 tebibytes. However you say it, it is a lot.

Btrfs doesn't contain any database-specific optimizations, and is not a clustering filesystem. It is designed to handle very large storage volumes, protect data, simplify large storage management, and read and write fast.

A COW – copy on write – filesystem is extra-careful with writing your data. When you make a change to a file, the old data are not overwritten. Instead, the filesystem allocates new blocks for the new data, and only the changed data are given a new allocation. The downside is this creates fragmentation. So Btrfs supports online defragmentation with the

btrfs filesystem defragment

command.

COW filesystems lend themselves to easy, efficient snapshots, and Btrfs supports both snapshots and rollbacks. The easy safe way to try Btrfs is to create a new partition for testing. Gparted supports Btrfs, as you can see in figure 1.

Next, mount this partition. In this example the mountpoint is /btrfs-volume:

# mount -t btrfs /dev/sda8 /btrfs-volume

Now we can create a subvolume in this partition. Subvolumes are cool. They are like independent filesystems inside the parent filesystem, with their own mountpoints and options. Create one this way:

# btrfs subvolume create btrfs-volume/test

And that's all there is to it. You'll see this as an ordinary directory in your file manager (figure 2). You don't need to worry about allocating space like you do with normal disk partitions, because subvolumes automatically snag whatever space they need from the parent volume as you add data to them. So you can go ahead and copy some files into the test subvolume. You'll need root permissions, or you can futz with the file permissions in the usual way and change them to an unprivileged user.

Now let's create a snapshot:

# btrfs subvolume snapshot btrfs-volume/test btrfs-volume/test-snapshot-1Create a snapshot of 'btrfs-volume/test' in 'btrfs-volume/test-snapshot-1'

Snapshots are very efficient because multiple snapshots share the same original files and copy only the changes. You can list all the snapshots in the same volume; you need to name one of them and then all of them are displayed:

# btrfs subvolume list btrfs-volume/testID 256 top level 5 path testID 257 top level 5 path test-snapshot-1

This also shows that Btrfs sees snapshots and subvolumes as the same things. Your snapshots can be copied elsewhere as backups, or mounted independently to different mountpoints. Want to roll back to an earlier snapshot? First set the snapshot as the default. You need the snapshot ID, and then the path:

# btrfs subvolume set-default 257 btrfs-volume/

Then unmount the subvolume, and then remount:

# umount btrfs-volume# mount -t btrfs /dev/sda8 btrfs-volume

Is that not cool? After creating subvolumes you don't need to mount the parent volume.

Btrfs is still rough around the edges, and the documentation and administration tools are incomplete. If you've used ZFS then Btrfs feels like a clunky copy, because administering ZFS is faster and easier. ZFS has a several-year head start on Btrfs, though. I expect Btrfs will improve rapidly as it becomes more widely used.

Learn more about Btrfs with The Linux Foundation's free Linux training tutorial Introduction to Btrfs. A full course schedule of Linux SysAmdin training is also available.

Comments (4)
You must be logged in to post a comment. Please register if you do not have an account yet.

View the original article here

Weekend Project: Learning Ins and Outs of Arduino

Arduino is an open embedded hardware and software platform designed for rapid creativity. It's both a great introduction to embedded programming and a fast track to building all kinds of cool devices like animatronics, robots, fabulous blinky things, animated clothing, games, your own little fabs... you can build what you imagine. Follow along as we learn both embedded programming and basic electronics.

Arduino was invented by Massimo Banzi, a self-taught electronics guru who has been fascinated by electronics since childhood. Mr. Banzi had what I think of as a dream childhood: endless hours spent dissecting, studying, re-assembling things in creative ways, and testing to destruction. Mr. Banzi designed Arduino to be friendly and flexible to creative people who want to build things, rather than a rigid, overly-technical platform requiring engineering expertise.

The microprocessor revolution has removed a lot of barriers for newcomers, and considerably speeded up the pace of iteration. In the olden days building electronic devices means connecting wires and components, and even small changes were time-consuming hardware changes. Now a lot of electronics functions have moved to software, and changes are done in code.

Arduino is a genuinely interactive platform (not fake interactive like clicking dumb stuff on Web pages) that accepts different types of inputs, and supports all kinds of outputs: motion detector, touchpad, keyboard, audio signals, light, motors... if you can figure out how to connect it you can make it go. It's the ultimate low-cost "what-if" platform: What if I connect these things? What if I boost the power this high? What if I give it these instructions? Mr. Banzi calls it "the art of chance." Figure 1 shows an Arduino Uno; the Arduino boards contain a microprocessor and analog and digital inputs and outputs. There are several different Arduino boards.

You'll find a lot of great documentation online at Arduino and Adafruit Industries, and Mr. Banzi's book Getting Started With Arduino is a must-have.

The world is over-full of useful garbage: circuit boards, speakers, motors, wiring, enclosures, video screens, you name it, our throwaway society is a do-it-yourselfer's paradise. With some basic skills and knowledge you can recycle and reuse all kinds of electronics components. Tons of devices get chucked into landfills because a five-cent part like a resistor or capacitor failed. As far as I'm concerned this is found money, and a great big wonderful playground. At the least having a box full of old stuff gives you a bunch of nothing-to-lose components for practice and experimentation.

The Arduino integrated development environment (IDE) is a beautiful creation. The Arduino programming language is based on the Processing language, which was designed for creative projects. It looks a lot like C and C++. The IDE compiles and uploads your code to your Arduino board; it is fast and you can make and test a lot of changes in a short time. An Arduino program is called a sketch. See Installing Arduino on Linux for installation instructions.

You will need to know how to solder. It's really not hard to learn how to do it the right way, and the Web is full of good video howtos. It just takes a little practice and decent tools. Get yourself a good variable-heat soldering iron and 60/40 rosin core lead solder, or 63/37. Don't use silver solder unless you know what you're doing, and lead-free solder is junk and won't work right. I use a Weller WLC100 40-Watt soldering station, and I love it. You're dealing with small, delicate components, not brazing plumbing joints, so having the right heat and a little finesse make all the difference.

Another good tool is a lighted magnifier. Don't be all proud and think your eyesight is too awesome for a little help; it's better to see what you're doing.

Adafruit industries sells all kinds of Arduino gear, and has a lot of great tutorials. I recommend starting with these hardware bundles because they come with enough parts for several projects:

Adafruit ARDX – v1.3 Experimentation Kit for Arduino This has an Arduino board, solderless breadboard, wires, resistors, blinky LEDs, USB cable, a little motor, experimenter's guide, and a bunch more goodies. $85.00.9-volt power supply. Seven bucks. You could use batteries, but batteries lose strength as they age, so you don't get a steady voltage.Tool kit that includes an adjustable-temperature soldering iron, digital multimeter, cutters and strippers, solder, vise, and a power supply. $100.

Other good accessories are an anti-static mat and a wrist grounding strap. These little electronics are pretty robust and don't seem bothered by static electricity, but it's cheap insurance in a high-static environment. Check out the Shields page for more neat stuff like the Wave audio shield for adding sound effects to an Arduino project, a touchscreen, a chip programmer, and LED matrix boards.

Let's talk about volts (V), current (I), and resistance (r) because there is much confusion about these. Volts are measured in voltage, current is measured in amps, and resistance is measured in ohms. Electricity is often compared to water because they behave similarly: voltage is like water pressure, current is like flow rate, and resistance is akin to pipe diameter. If you increase the voltage you also increase current. A bigger pipe allows more current. If you decrease the pipe size then you increase resistance.

Talk is cheap, so take a look at Figure 3. This is an old circuit board from a washing machine. See the stripey things? Those are resistors. All circuit boards have gobs of resistors, because these control how much current flows over each circuit. The power supply always pushes out more power than the individual circuits can handle, because it has to supply multiple circuits. So there are resistors on each circuit to throttle down the current to where it can safely handle it.

Again, there is a good water analogy — out here in my little piece of the world we use irrigation ditches. The output from the ditch is too much for a single row of plants, because its purpose is to supply multiple rows of plants with water. So we have systems of dams and diverters to restrict and guide the flow.

In your electronic adventures you're going to be calculating resistor sizes for your circuits, using the formula R (resistance) = V (voltage) / I (current). This is known as Ohm's Law, named for physicist Georg Ohm who figured out all kinds of neat things and described them in math for us to use. There are nice online calculators, so don't worry about getting it right all by yourself.

That's all for now. In the next tutorial, we'll learn about loading and editing sketches, and making your Arduino board do stuff.

Comments (0)
You must be logged in to post a comment. Please register if you do not have an account yet.

View the original article here