This aspect of development, called Documentation, is the source of various frustrations. It’s hard to get do it, but why? My feeling is that it’s like testing. When you begin your craft as a coder, all what matters is the code. It’s only after some iterations that some non-code aspects come back bite you in the neck. Like, huho, now that I need to refactor, I really should use something that tells me if all still works. Tests become an early necessity to anyone who knows that kind of truth. If you wait until the end of a coding cycle to write those, the task is huge and it cuts you from your productivity cycle. Write them early, along the flow, is way easier.

So I think documentation follows the same pattern. So many software projects are badly documented because this aspect is postponed until it’s needed. Means at release stage. In early stages, you work on a prototype and you don’t need to explain how things work or are supposed to work. And when it’s released, there is usually some other task waiting and it’s hard to stop everything to get back and document things properly. It may be a flaw in the agile process, but it may also be a feature: if you don’t document along the way, you won’t document much.

Personally I try to consider the documentation of any of the source code I write as one of the first tasks. There are various tricks that can help in that perspective, like readme-driven development, or including the doc inside the code with yarddoc or apipie, or coupling documentation with tests with rspec or RSpec API Doc Generator. But honestly I prefer edited documentation that can follow a structure that is thought as documentation rather than merely an automated output of some code.

There are various tools to organize edited documentation, like ReadTheDoc which is in python. Another project appeared last week from the guys at Pluralsight named hack.guide(). That’s a community project but actually being totally open sourced, I was thinking the documentation building CMS they made could pretty much fill the gap of a light-weight RTD, with the benefits of having a UI for editors. Too bad we don’t have anything like this in ruby (or do we?).

Recently I had to package a gem for debian as a .deb. As a matter of fact, there is a lot of companies that are not in the ruby ecosystem and they rely on things they trust rather than rubygems. For a while debian was way behind on the inclusion of ruby and versions were outdated there. So a parallel distribution ecosystem had to be created. Now we have rvm, rbenv, or chruby for ruby and bundler and rubygems for the packages. But where I work, they package everything as debian packages. So I had to adapt.

So I went have a look at the debian ruby team and checked out the tools. That gem2deb tool is pretty convenient. I am using debian for a while but never really had to package anything. The debian community process is a bit complicated and they are not very inclusive. Compared to the ease of uploading any gem to rubygems, it’s quite a fortress, and people in the process are not very welcoming.

Maybe it’s generational. Debian started before sourceforge existed, they are rooted in a community tradition that are very exclusive and a bit elitist. But technology changes faster than people, obviously. Younger communities bootstrap in a more fluid environment.

I’m not sure it is related but maybe it is, there was another epic [Linus Hit][liushit] recently. Well, all those things put in perspective the niceness of the ruby community. Really, cherish it and enjoy it, it’s precious.

While I was playing with react.rb and exploring opal, I felt in a totally different world. The power of the DSL already prove its shine on various occasions. I got to experience it in activeadmin, rspec, and puppet (but I kind of dislike the one in puppet).

As I’m also writing some python, I can see a real difference in expressibility that can be found in ruby. Really, ruby is a maker tool. Like a toolbox for making new programming languages, using meta-programming, lambda and other goodies. This is what, in my opinion, makes ruby unique. Beyond the elegance of its syntax, it empowers developers with the ability to write dedicated programming language for any specific use. And this, I didn’t find in python.

This is a bit twisted. In ruby, as you know, you cannot use multiple inheritance. But the flexibility of the language, especially the metaprogramming aspect, opens the door for some cheating. Check out this code, it tricks it by declaring the first class as the super class and the remaining classes as modules included in that super class. Pretty acrobatic and elegant, imho.

About vaping

Well this has been my burden for 30 years, and I’m not proud of it. But last week it finally ended. I quit smoking. But this was kindof a trick, as it was made possible by switching to vaping. It certainly doesn’t solve the nicotine addiction issue, but I’m very happy with this move and hopeful for a gradual freedom.

I also found out that this vaping practice is very geeky. The hardware part is quite elaborate, vaping devices are really neat piece of work, there are a bunch of vendors. The liquidware is also quite a subject of hacking. Technically it’s not that hard to do it yourself. Not surprising that I discovered this vaping practice in geeks circles.

Well, since that whole blogging thing began, I never have been very active on it. Well, I have a blog on Tumblr because I wanted to know how they are doing it, I published various posts in the faria devtips, and after all, this rant could also count as a publication. So I think I will gather them all under one unique site. A Jekyll github-pages kind, easy and cheap.

After all, I’m not sure the devtips website will stay up any longer. There have been no post since the day I left. Too bad. It’s a demonstration that some collective actions can sometimes rely only on the energy of one person.

So, I made a new repo on github for it, and I will gather whatever stuff I can find that I wrote in there.

Lovely FreeBSD

At our Gandi office in Taipei I had to install a pfsense server, which is based on FreeBSD. It was quite a pleasant experience, actually. Last time I played a bit with BSD that was 12 years ago, and that was not very smooth. I’m happy to be given that occasion to see how it goes now.

For now I’m going to use it like if it was an OpenWRT with some extra OpenVPN abilities. And it will also be a file server for the LAN. Not sure yet how I will handle that.

Ruby package management

When I got in my new job, I discovered a new way to manage server management. They didn’t want to use rvm, or even ruby gems, or pip or anything that is not debian packages. It may sound quite harsh. Since I came in ruby in 2010, rvm has been my best friend, bundle the second one. But this approach is very developer-based. When you maintain large and stable systems, it’s more likely that you will not trust the bleeding edge stuff and prefer confirmed publication of packages before using them. This is an interesting slap on my face.

Of course there are the brightbox packages for Ubuntu, but that’s missing the point. A release has to be out for a certain time so it can be strengthened by security reviews and proper production usage. There is a part of the population for each language that is considering the instability of current releases a normal trade-off. but there is a huge lot of other companies that will wait patiently that things get stable enough for their taste.

Our craft as webdev is clearly in huge demand. This provides an opportunity for us to travel abroad, discover new cultures, widen our horizon. When I went to live in Taiwan, I was not worried about finding a job. But when hunting for a job abroad it can be tricky to find companies that will sponsor a visa, a work permit, or whatever legal requirement is involved in each countries.

That’s what the new initiative called TechMeAbroad aims to fix, by listing only job opportunities for which a work permit is sponsored. As Julien Barbier explains, it will launch in march, and if you know of any job offers that includes a visa or such thing, please let him know (or send it to me I will pass it along), so we can feed that platform with some real data quickly.

So long ruby 1.9

As planned, it’s now time to say byebye to ruby 1.9.3. But there is still so many old version of ruby in the wild. In debian squeeze, you gotta use 1.8. In wheezy, you have 1.9. And there are still many servers that will refuse to upgrade.

Pick your hero

The registration for Ruby Heroes 2015 is now open. Go vote for your favorite ruby influential dev. You have until railsconf that will happen in april, in Atlanta this year.

While42

This week I got in contact with Sylvain from while42. It’s a social network for french tech engineers, but with a special touch on it. There is no facebook page or google group. They intentionally decided to be irl-first. For people that are most likely to be online all the time, this sounds so therapeutic! So, I decided, with Thomas from Gandi, to setup a Taiwan chapter. If you want to know more (and are french, engineer, and living in Taipei) just drop me a mail.

Traveling ruby

This is a project from the Phusion people, called Traveling ruby. Its purpose is to make it easy to build auto-sustaining tarballs for ruby programms, including ruby binary and all gems in it. That way you can publish a tarball and your customer/user will just launch whatever you put in it. No need to install ruby or any gems.

I gave it a try yesterday on my zabbix-ruby-client gem, and after a few tweaks (mostly because I wanted to use an unpublished version of my gem), it all worked pretty fine. Result is a tarball of 8M. Uploaded to the server (an old squeeze), it just ran the self-contained ruby 2.1.5 with no hassle. It reminded me a bit of the feeling when you upload a go executable on the server. Really neat.

Palm control

There are tiny things that can change a lot. My recent tiny thing is the palm control technique. Every sunday when I prepare Green Ruby I do a huge amount of copy pasting all over the place. Recently I found on a post about keyboard someone talking about the palm control technique for avoiding the copypaste fatigue.

Well, you need to have a keyboard that makes it possible. Mine is a Logitech washable k310. I already liked its look, and even better, it’s perfect for this palm thing because keys are prominent. Use the palm to push control and then it’s only a matter of hitting c or v with the index. Honestly, it changed my life!

We use S3 to backup various kind of files on MB. We use the very convenient backup gem for that (we still use 3.9.0).

• http://meskyanichi.github.io/backup/v4/

But at some point it appeared that backing up our audio recording was hammering disk IO on our server, because the syncer is calculating md5 footprint for each file each time a backup happens. When you get thousands of big files that is pretty expensive process (in our case 20k files and 50G total).

So I added a small trick there:

in Backup/models/backup_audio.rb

module Backup::Syncer::Cloud
  class Base < Syncer::Base
    def process_orphans
      if @orphans.is_a?(Queue)
        @orphans = @orphans.size.times.map { @orphans.shift }
      end
      "Older Files: #{ @orphans.count }"
    end
  end
end

Backup::Model.new(:backup_audio, 'Audio files Backup to S3') do

  before do
    system("/Backup/latest_audio.sh")
  end

  after do
    FileUtils.rm_rf("/tmp/streams")
  end

  ##
  # Amazon Simple Storage Service [Syncer]
  #
  sync_with Cloud::S3 do |s3|
    s3.access_key_id     = "xxx"
    s3.secret_access_key = "xxx"
    s3.bucket            = "bucket_backup"
    s3.region            = "us-east-1"
    s3.path              = "/mb_audio_backup"
    s3.mirror            = false
    s3.thread_count      = 50

    s3.directories do |directory|
      directory.add "/tmp/streams"
    end
  end
end

and in Backup/latest_audio.sh

#!/bin/sh
# isolate files changed in the last 3 days

TMPDIR=/tmp/streams

mkdir $TMPDIR
for i in `find /storage/audio/ -type f -cmin -4320`; do
  ln -s $i $TMPDIR
done

It creates a fake backup dir with links to the files that actually changed in the last 3 days and patches the syncer to avoid flooding the logs with orphan files. When sometimes S3 upload fails on one file (and it happens from time to time for ‘amazonian’ reason) it will be caught on the next daily backup.

The result was pretty obvious on our disk usage with our daily backups:

in the logs:

[2014/06/10 07:00:25][info] Summary:
[2014/06/10 07:00:25][info]   Transferred Files: 5
[2014/06/10 07:00:25][info]   Older Files: 22371
[2014/06/10 07:00:25][info]   Unchanged Files: 16
[2014/06/10 07:00:25][info] Syncer::Cloud::S3 Finished!
[2014/06/10 07:00:25][info] Backup for 'Audio files Backup to S3 (backup_audio)' Completed Successfully in 00:00:22
[2014/06/10 07:00:25][info] After Hook Starting...
[2014/06/10 07:00:25][info] After Hook Finished.

I have been looking for a way to see what gem uses a gem, so I can see examples of integration in other projects. The rubygems API guide don’t tell anything about such reverse dependency query. But it is actually there, it got merged some time ago, and optimized, but it is not documented yet (it runs really fast, kudos Rubygems guys).

ruby -ropen-uri -rpp -ryaml \
     -e 'pp YAML.load(open("https://rubygems.org/api/v1/gems/rails_best_practices/reverse_dependencies.yaml"))'

["metric_fu",
 "flyerhzm-metric_fu",
 "edouard-metric_fu",
 "devver-metric_fu",
 "goldstar-metric_fu",
 "socializer",
 "trollface",
 "guard-rails_best_practices",
 "rferraz-metric_fu",
 "git-hooks-helper",
 "odor",
 "rake_check",
 "koality",
 "danmayer-metric_fu",
 "bf4-metric_fu",
 "metrics_satellite",
 "code_hunter",
 "kinit",
 "rails-audit",
 "pronto-rails_best_practices",
 "free_disk_space",
 "warder",
 "ruby_osx_app",
 "sanelint"]

Out of curiosity I counted some wellknown gems usages by adding a .count at the end:

rake:    23766
rails:   6283
thor:    2786
pry:     2870
sinatra: 1964
devise:  422 

In Faria we use hackpad a lot, it’s pretty useful. Mose made a command line interface so he can download all the pads locally and grep them all in one go. It also transforms the markdown much better than the original hackpad markdown export, which totally sucks (as for now). So pads can be recycled easily in github wiki pages.

• https://github.com/mose/hackpad-cli