Emacs 23 Tramp: sudo to another user account on a remote machine

It took me a while to figure out the correct incantation to log in to another machine and change user account with sudo: `sudo -u someuser -i`. It’s actually very easy once you know how:

.emacs

(set-default 'tramp-default-proxies-alist
(quote (("remote.domain" "someuser" "/ssh:%h:"))))

In emacs:

C-x C-f /sudo:someuser@remote.domain:/some/dir

There’s some nice code on http://www.gnu.org/software/tramp/ to display the hostname in the mode-line:

(defconst my-mode-line-buffer-identification
(list
'(:eval
(let ((host-name
(or (file-remote-p default-directory 'host)
(system-name))))
(if (string-match "^[^0-9][^.]*\\(\\..*\\)" host-name)
(substring host-name 0 (match-beginning 1))
host-name)))
": %12b"))

(setq-default
mode-line-buffer-identification
my-mode-line-buffer-identification)

(add-hook
'dired-mode-hook
'(lambda ()
(setq
mode-line-buffer-identification
my-mode-line-buffer-identification)))
Advertisements

DIY Conference Video using Free Software

For the recent Plone conference in Munich we decided to try to do the video ourselves. The primary motivation was to save some money since the professional video companies which were contacted requested several thousand Euro (which is perfectly reasonable for covering 30 talks) but our budget was limited. We also wanted to get the video online as soon as possible but no conventional video companies were able to offer this since they required additional time to do all the typical post production work. We decided to use the same process that Debian have been using for their conferences for years and has been adopted more recently by PyCon US and FOSDEM among many others.

DVswitch

The key component in this setup is DVswitch: ” a digital video mixer intended for interactive live mixing of several incoming DV video streams”. This is really simple to use and allows someone without any experience in video to do the editing while the talk is taking place. This cuts out the need for lengthy post-production. It is even possible to stream this video live although we decided to keep things simple for our first attempt and just record the video to be uploaded at a later stage. As the editor you can choose to display the output of the laptop fullscreen, the output of the camera fullscreen, or you can draw a rectangle anywhere on the screen to display the output of one or the other (picture-in-picture, see illustration 2 below).

Hardware

We needed to cover two rooms and in an effort to keep the setup as simple as possible we decided to just use one laptop, camera and mixing desk per room. I learnt two important things about Firewire when setting up and configuring the systems. First of all, it is only possible to use one video device per controller. We were lucky that both of our laptops had Firewire inputs as well as expansion slots which allowed us to add a Firewire card to each laptop giving us the two controllers we needed per laptop (the two additional Firewire ports on the card could not be used). Otherwise we could have used two laptops per room and connected them via Ethernet. This is actually the more typical configuration. The other thing I learned was that you should never connect a Firewire device while it is turned on! I was trying to figure out why a particular camera would not work consistently with the system when I learnt about this. Luckily nothing was damaged, but if you search for Firewire Hotplug Damage you’ll quickly find many horror stories. Admittedly, all the stories I came across were from Mac users so maybe “It’s a Mac thing”®, but I have no intention of testing this 🙂

The cameras we used were good old Sony DV cameras, a VX2000 and a VX2100. These are really nice cameras with great low light capabilities and plenty of manual control. The most important thing was that they have Firewire outputs and Mic inputs (mini-jack rather than XLR). These worked without a hitch, unlike a modern HD camera which we borrowed to test which occasionally would just stop working and needed to be power cycled to start working again.

Laptops and cameras are pretty easy to come by, but the other piece in the puzzle is a frame grabber. This takes the VGA output of a laptop and digitizes it so that it can be sent as a DV stream over Firewire to the laptop, it also sends the VGA output to the projector. Many thanks to the good folks on the DVswitch and PyC0n-AV mailing lists for guidance and advice on the available options. We stuck with the Canopus TwinPact 100 which is what most people use. It’s not cheap at over €500 per unit, but we decided it would be a worthwhile investment since we can use it again in the future.

Connecting it all together is quite straight forward. The diagram below is not exactly to scale so it’s important to note that the camera, TwinPact and laptop have to be pretty close because Firewire cables can’t be very long (~5M is OK). VGA and XLR/audio cables can be very long. For one talk we joined two 10M VGA cables with a connector and it worked perfectly. The audio cable was 25M in one room!

True story

Staging

If the language of the conference is written from left to right, then the ideal configuration is to have the projector screen to the right hand side of the speaker (as the speaker faces the audience). This often leaves a neat little corner on the bottom right of the screen for inserting the video of the speaker. You might also be able to throw some extra light onto the speaker without it washing out the image on the projector screen.

One of those magical moments when a presenter is working through their bullet points and pointing with enthusiasm at the projected image on their right. Alas, the camera operator was anticipating more subtle emotion and zoomed in for facial detail when the presenter made a sudden and unexpected gesture just out of frame. "Is fada an bóthar nach mbíonn casadh ann".

Hiccups

One of the laptops was bought with the intention of using this system again for other events. This hadn’t been heavily tested and we were unfortunate to run into an issue with the Intel graphics module, Ubuntu and Gnome 3. Switching to the 2D Unity desktop helped considerably, although we did have further issues which seemed to require a reboot once a day. The other laptop was until recently my primary development machine (thanks Syslab!) so I knew I could trust it and I set it up myself. NixOS is my distro of choice so I used a simple config with just XFCE, DVSwitch, Firefox, Pidgin and VLC. Ancient proverb: “If it’s not tested, it’s broken”.

Audio was a constant concern. We noticed a hum from the PA which we couldn’t do anything about, and there was considerable hiss on the other system. As the conference progressed we got a bit sloppy with the cables and at one stage an unshielded audio cable (Mixer -> Camera) was running straight along a power cable!! That produced a disturbing ground loop hum which couldn’t be totally cleaned up in post production. Apart from that, there was a bit of feedback from the PA system dependant on the position of the speaker, and mobile phones and UMTS/3G dongles occasionally played havoc.

There is really only one button on the TwinPact control which you need to press. It’s the Overscan button. Sometimes, (it only seemed to happen with Apple laptops, but not with all of them) the picture from the TwinPact would go crazy even though the image from the projector was fine. Pressing the overscan button caused a thick border to be displayed around the image but apart from that it was fine.

Apart from these minor issues it was a roaring success. There is a palpable fear in the Plone community right now that some of our brightest stars will be talent spotted on YouTube and seduced into signing deals with major movie studios, much to the detriment of the Content Management Systems of the future. If you are a such a talent spotter please don’t look at the videos.

TODO

Post-production notes (maybe a separate post):

Gimp: titles

Kdenlive: in-out points and transcoding

Audacity and ffmpeg: audio cleanup

Credits:

The spectacular AV-Team: Geraldo, Hans-Jürgen,  Sonnja, Toni. (I will add links)

The people who made it possible: Max, Patrick, Philip.

Everyone else and also http://www.ragemaker.net and the rage comic community for making enterprise grade illustration material attainable to everyone in a world gone off the rails.

The Free Software business model … and how it could save the world?

To me, the most interesting Free Software business model is where people who are interested in a project contribute money directly to the people who work on it. Some equate this with “Tipping” and don’t see it as sustainable. I’m inclined to believe it’s something quite different.

From time to time I daydream about lofty topics such as politics and economics. Fields I have no qualification in, and truly know very little about. I shape my opinions from random chats on irc and the occasional stroll around wikipedia along with the odd series of lectures generously provided by universities as podcasts. In rough terms, my understanding is that the Left aims to protect the weak by distributing wealth and resources and the Right aims to protect the strong by letting everyone fight for their own position.

The Left claims to support the cause of the downtrodden Working Class. According to the literature, the Working Class are kept in their place by the rich elite who pay the workers as little as they can. The logic goes that if people were to organise themselves there would be no need for this upper class at all because the resources would be pooled. This would greatly improve efficiency since there would be no need for “wasteful” competition. There would be so much more to go around that the majority would have to work less and less and be able to enjoy all the benefits of the shared effort in their free time.

The Right claims that people are hampered by high taxes which are being wasted to keep l
azy people comfortably housed and clothed. I guess they also believe that there is some kind of natural order in society and that without wealthy people ruling over the lazy masses there would be chaos. I often wondered why the right has such a strong following among poorer people, since it would seem that they stand to benefit the least from a hierarchical society ruled from above. I can only conclude that the leaders of the Right just take advantage of the poor education and desperate situation of the poor and manipulate their fears and prejudices to gain support.

Central politics seems to be what people mostly vote for and presumably want. People want to have some security i.e. they won’t starve if they loose their jobs and that they will be able to see real benefit in proportion to the amount of effort they put into their careers. Naturally, some people abuse the social welfare system to be lazy and some abuse the economic and legal system to get rich. Both at the expense of the majority of people who contribute to the system.

We’re in an interesting economic situation at the moment which seems to have left people feeling helpless, and misled. Not so long ago, everyone was going crazy trying to invest in property. I looked into it myself too, but luckily I found the idea of paying so much interest to the banks for such a long time unappealing. If I hadn’t been so stingy I would have paid for a flat with money I didn’t have, and at this stage I’d probably be bankrupt, and the taxpayer would be paying off my debt to the banks so that they don’t lose their own savings. But it begs the question, who’s to blame? An awful lot of normal people participated in the mania, driving the prices up as they went. It’s all very well to blame the bankers, but if they didn’t have such eager customers things wouldn’t have gotten so exciting.

I love the welfare state because it gives us a reasonably fair shot at getting any position we want in society through hard work, and we are supported if we are unlucky or unsuccessful. We have a (very small) say in the political system but we are free to choose how we spend most of our money and therefore responsible for the businesses we support. If everyone took their money out of the banks and put it into credit unions instead then the banks would go out of business and that would be that.

Some say the greedy rich are to blame and we just need to tax them more to even things out. There is no doubt that they’ve been making the most of the situation and rewarding themselves with generous bonuses to boot. Capitalist economists argue that this will cause growth to slow because there will be less capital available for investment. The idea here is that the small group of ridiculously rich people have that money because they are talented at making money i.e. they invest in “successful business ideas” which work, create jobs (as a side-effect) and allow them to accumulate more wealth. Since these people create jobs for the masses, the wealth they generate trickles down eventually. Capitalist economists believe this is how it should be. Perhaps they would concede to some regulations here and there to make the act of “making money” be somehow related to providing real jobs rather than simply playing a game in which they make the rules, which they can’t lose, with our money.

So that’s about as far as my understanding goes of how our politics and economics work. Now for Free Software business models as I understand them. There are quite a few and some have already proved themselves well. For niche custom applications it seems to make a lot of sense to share a core set of libraries freely and improve it collaboratively since the developers can include improvements to the core system as part of the cost of creating the custom applications. Everyone wins, everyone gets paid, the solution the client has invested in is constantly improved, they have a selection of providers etc. The Plone Content Management System is a nice example of such a project that is very close to my own heart (and wallet).

Other business models that are working well for free software are the sale of secondary services such as customer support, and advertising. While these do generate revenue, it’s not really very exciting. The less established business model which interests me relates to applications which are consumer oriented. Why would a consumer pay for something which they can have for free? Why would a developer ever work on some boring application that needs to be updated regularly e.g. for doing tax calculations? Many projects have experimented and the voluntary subscription/donation business model seems to be having some success. Ardour (Digital Audio Workstation) earns enough to pay the lead developer to work on it full-time (although not nearly as much as he deserves) and Mandriva (Linux Distribution) has been doing something like this for many years. Blender (3D) has it’s own very interesting spin on this, the project asks the community to regularly pay for a movie or game before they create it, the money is invested directly into the improvement of the software so that this movie or game can be realised. The latest project, Sintel is only a few months away from being released, and the whole community is very motivated and excited about it, watching every step and even helping create some of the content in a series of sprints. Recently http://www.wolfire.com/humble had such enormous success by allowing people to pay as much as they wanted for a collection of their games that they decided to release the source code for them.

The implications of this on the economy could possibly be enormous. By donating to these projects you aren’t “giving away” to a charity to improve the lives of the less fortunate. You are in fact investing in the improvement of the services which effect you directly, as well as everyone else. Through the magic of the internet the masses have become the investors. I see no reason why this should only apply to software. There are open hardware projects e.g. Arduino and lots of creative commons music e.g. Jamendo, Dogmazic. Masses of people could just as easily invest in big-budget movies, games, space ships, whatever.

What is in it for people to invest in something they could get for free? Surely the greedy, who take whatever they can get and give nothing back stand to benefit from such an economy. Could it be that it would be embarrassing to be noticeably more wealthy than your peers because it insinuated you didn’t contribute enough? The less you contribute means the less influence you have, and it’s just much less fun. The Blender example is a particularly good one. It’s such an incredible joy to feel part of the process. To know that because you and enough other people contributed a little money there’s a team of cool people working really hard at something they love, creating something amazing and beautiful to share with the whole world. It’s just not so much fun to pay for a cinema ticket or even to download illegally without paying at all.

Such a culture would have no reason to represent the greedy in an appealing manner. Culturally, the greedy would be worthless and perhaps that alone is enough of an incentive. Could it go further? Would people invest in the social welfare system or health system etc. voluntarily instead of being forced to pay tax? What’s the big difference? Through hard work and violent struggle our ancestors created a free society for us to inherit. They didn’t create taxes to oppress us, they created taxes to support us. We created taxes for ourselves, why don’t we take the next step and pay as much “tax” as we want by financially supporting the services we want?

There is a common notion, especially among societies based on the Judeo-Christian-Islam belief system, that people are born evil and greedy. They require threats and punishments and laws to keep them from doing harm to the whole group. Increasingly, there is a drive to monitor more and more of each others’ activities to ensure nobody slips up and does something evil. This is a very efficient belief system for a hierarchical society. Those at the top can conveniently manipulate the legal system and buy their way out of any difficulties. They can avoid taxes and to a large extent influence the laws regarding taxes and regulations to suit their business interests. In a way, the society at large accepts this, since this upper class are considered important for everyone’s welfare. This is a great system for imperialist nations. The masses can effectively be ordered to kill each other to strengthen the power of those above them. This system has undoubtedly proved its success throughout the world and even though most people are ashamed of the gruesome methods their ancestors employed, people generally accept this system as the way things are meant to be.

Part of what makes this hierarchy so effective is the control of information. By having privileged access to “holy books” and the education required to sell these ideas to the masses, the religious hierarchy had a distinct advantage over them. If a large group of people consistently speak of the exact same fairy tales with a great sense of authority  it makes it increasingly difficult to believe otherwise, regardless of how crazy it is, or what rules they impose on you as a result. Once the system is in place to control the information, it is easy to tweak it to the interests of those at the top.

The internet has changed everything. Information about how the world works no longer trickles down via the clergy or wealthy publishers. They are feeling the pinch now as people increasingly turn to blogs, podcasts, wikipedia etc. as sources of information. We already knew that we could not blindly trust the media. Now we are also perfectly aware that we cannot trust what we find on the internet. Whether a sock-puppet actually has a piece of paper from a university to qualify their “opinion” or is published in a “respectable” source matters less and less. It is up to us to process the information we receive and decide who and what we trust. Even the media companies which aren’t obviously under the corporate thumb find themselves under increasing pressure to mix advertising and agenda into their content in order to survive. They claim they don’t have the resources to spend on real journalism anymore and are more inclined to accept what they are fed as information. The more they do so, the more people will come to see that this “professional” media is worth less than the media and opinions which are shared freely.

At the moment the system is broken. The funny thing is that people like to support and show their support for things that they are interested in. Media corporations, however, are founded on the notion that there is such a thing as “Intellectual Property” and that if you don’t charge people for it before you allow them to use it (under very specific terms and conditions) then you won’t be able to pay people to make more of it. They believe they are protecting the general public interest by enforcing regulations which ensure they will have money to produce more of what (and they know best) people actually want. Sure, people like the kinds of movies that require thousands of people to dedicate significant time, often years, to produce. People want to see these movies. Why not facilitate people to invest in these movies too? So far this model has worked on some independent movies, but it has yet to be attempted with a big budget movie. Instead of market researchers trying to guess what people want, advising a small number of investors, who then do their best to influence the movie to the apparent desire shown in statistical analysis of public opinion, the whole system could work the other way around.

There could be a whole new sector dedicated to facilitating the public’s desire to invest. The products of the work could be distributed freely, since the people who worked on creating it have already been paid. People involved in successful projects would be justly rewarded and be chosen for future more ambitious projects. People could be allowed to freely reuse the work, since this cultural “property” wouldn’t need to be “protected”. Instead of creating jobs in law enforcement and punishment we could be creating jobs in facilitation and communication.

The free software movement didn’t invent this business model, e.g. Pacifica radio has been financed by its listeners since shortly after it was founded in 1946. There’s nothing new about people pooling resources to create something bigger than they could on their own. If the media companies aren’t willing to take a leap of faith (even though many commercial artists such as Radiohead and Nine Inch Nails already have), then the free software movement is in a great position to show how it works, while also providing the infrastructure to make it happen.

I am excited and optimistic that such an economy and society is naturally taking shape. I see no need for a class war, or segregation, or violent revolution. All we need to do to make it happen is to invest in it.

Naïve? Loopy? Freetarded? Heard it a million times before?

What are your thoughts?

IE 7 and IE 8 on Linux with Wine 1.1.20

In an interview with Jeremy White (C.E.O. of CodeWeavers) on The Linux Action Show he mentioned they have IE7 working in CrossOver Office in the latest beta release of version 8 and that the latest version of Wine has lots of improvements that make this possible. I thought I’d try it out as I’m working on a new site and would like to be able to test it. I installed the latest version available for Mandriva 2009.0 in the package manager and did the following dance:

Created a fresh ~/.wine
Ran winetricks and set the version to win2k and installed IE6.
Ran winetricks and set the version to winxp.
Installed IE7:
$ wine IE7-WindowsXP-x86-enu.exe
Installed IE8:
$ wine IE8-WindowsXP-x86-ENU.exe
I think it installed the first time, but when I tried to run it, it threw an error. For no good reason I ran the installer again. After that it works fine (except it doesn’t seem to run any JavaScript). I think it is very usable for testing layout issues.

Some sites throw an error: “Internet Explorer couldn’t download %ws” and not having JavaScript working is a problem, perhaps I’m just missing a dll or something. I think it’s really great news though. I bet the CrossOver Office folks will have this polished pretty soon (if they don’t already).

It’s really phenominal work the Wine community do and it is greatly appreciated. I have no copy of Microsoft Windows and from time to time I really need to run a Windows application (e.g. Tax Software from the Government) and without Wine this wouldn’t be possible. Up until today I could only partially test in IE7 so it is great to know that I now have a way to make sure that sites I make are at least functional in Internet Explorer. Unless the site is for a client I’m not going to go to kill myself trying to make it work perfectly though, I tend to assume that people who use Internet Explorer are accustomed to things not working perfectly 😀

Goal oriented searching?

I was just wondering if there’s a way to search according to a particular goal. Since we “check for updates” to news items, activity on social networks etc. “read up on” topics in wikipedia, blogs etc. “look for” products, instead of just “searching” by keywords and trying to pick clever ones to get decent results wouldn’t it be nice to express our intention?

http://agents.media.mit.edu/projects/goose/ sounds like a proposal to do something like that, but doesn’t seem to have taken off, and at a glance seems overly complicated with some natural language processing functionality. Since you probably never want to “read about” something on ebay or “shop” on wikipedia is there a simple approach? Could it be that most sites serve a particular goal and would fit into a neat and small set of goals which would be easy to state? I guess something like http://rollyo.com might already have user created searches which could effectively do this, but it seems a bit too manual.

How about adding metadata to a page to indicate it’s purpose? Like keyword metadata maybe this would just be abused by SEOs until it is worthless. Hmm …

Are your modules ready for Drupal 6?

I was wondering when I could upgrade a Drupal 5 site to Drupal 6 so I knocked out a little script which reads the project page for each Drupal module you have installed and tells you the current releases and the version of Drupal that it can be used with:

http://drupal.org/project/og
Release: 6.x-1.0-rc6        Drupal version: 6.x
Release: 5.x-7.3           Drupal version: 5.x

It’s here:
http://cairde.net/social/drupal-module-status.py
(It requires Python and BeautifulSoup http://www.crummy.com/software/BeautifulSoup)

How to use a git hook to upload to a website that only allows ftp

NOTE: FIRST SEE THE UPDATE AT THE END FOR A NICER WAY TO DO THIS. USE AT YOUR OWN RISK, THIS WILL PROBABLY DELETE EVERYTHING ON YOUR SERVER AND EAT YOUR GRANDPARENTS!

There absolutely has to be better ways to do this, but with my weak bash-fu and lots of help from the good people on #git I stumbled through and got something working.

I am using gitosis to host a repo that is shared between a couple of people I trust. They have given me their keys and commit via ssh. I just followed the instructions on http://scie.nti.st/2007/11/14/hosting-git-repositories-the-easy-and-secure-way

I used yafc for the ftp part. It was my first time using it and I love it to bits. No more gui ftp clients for me, or plain old ftp either. The bookmark feature is awesome and really helped me out here.

  1. Login to your site via yafc anc save it as a bookmark
  2. Add the script below to your remote repo i.e. login to the server you are pushing to, if you set it up with gitosis you will have /home/git/repos/YourRepo and put it in hooks/post-receive.
  3. Finally make hooks/post-receive executable to enable it (chmod u+x post-receive)

#!/bin/sh

CHANGES=`mktemp -q -t`
FTPSTAGE=”/home/git/YourRepo-ftp/”

while read oldrev newrev refname
do
# This gets the paths of the files in the repo that have changed
# and shows if they are added A, deleted D etc.
echo `git diff-tree –name-status -r $oldrev $newrev` > “$CHANGES”
echo $CHANGES >> /home/git/YourRepo.log
UPLOADS=`cat “$CHANGES” | grep -v “^D” | cut -d” ” -f2`
# The sed expression here is to convert the path in the repo to the path on the server
DELETES=`cat “$CHANGES” | grep “^D” | cut -d” ” -f2 | sed -e ‘s/^drupal\/YourRepo.website\///g’`
echo `date` “\nUploading: $UPLOADS\nDeleting: $DELETES” >> /home/git/YourRepo-git-ftp.log
for U in $UPLOADS
do
# Again to change the repo path into the server filesystem path
FTPPATH=`echo “$U” | sed -e ‘s/^drupal\/YourRepo.website//g’`
# This dumps the files out of the repo on to the filesystem to be uploaded later
mkdir -p $FTPSTAGE`dirname $FTPPATH`
git cat-file blob $newrev:$U > $FTPSTAGE$FTPPATH
done
done

# Tell yafc to upload everything that has been dumped from the repo
# and delete everything that needs to be deleted
yafc <&lt;**
open YourBookmark
cd /public_html/YourRepo.website
put -fr $FTPSTAGE/*
rm -r $DELETES
close
**

# Tidy up
rm -rf $FTPSTAGE/*

If you’re any good at bash you are probably screaming in disgust right now but if you can spare a minute to explain how to improve this I would be grateful.

Note: this is so lame it doesn’t delete directories that have been created when you delete them from git. It just deletes the files. That’s just because the git command to get the paths doesn’t show when you’ve deleted the directory higher up. It just about works for me now, even though it’s horrible.

UPDATE: Replace all that gubbins with something like the following procedure:
Clone the repo on the server and set up sitecopy to sync that with the live site. Then in the post-receive hook:

cd /path/to/cloned/repo
unset GIT_DIR
git pull
sitecopy -u name

if you don’t change GIT_DIR you’ll see fatal: Not a git repository: ‘.’ even though running the hook on it’s own will work fine.
Thanks to: http://www.mail-archive.com/git@vger.kernel.org/msg03458.html