Capture Program Output on Python – subprocess.check_output()

Run an external program, capture output. With pipe support.

Prequisites: command line, Python hello world.

One Liner

Well, two lines if you count module import. Try it out in a python console:

$ python
>>> import subprocess
>>> subprocess.check_output("echo hello world", shell=True)
'hello world\n'

Yes, it’s that easy. You can also store the output in a variable.

Run Firefox, Run Anything

Obviously, you can also start graphical user interface programs from Python:

subprocess.check_output("firefox", shell=True)

Firefox starts.

Normal Pipes Supported

You can write normal shell pipes. This way, any command available in shell scripts becomes available in python.

>>> subprocess.check_output("ls /|grep bin", shell=True)
'bin\nsbin\n'

Caveats & Troubleshooting

Never trust user input. It’s best to avoid sending untrusted input to shell, or you are guaranteed to meet Bobby Tables. But if you start your python program from shell anyway, you can give yourself access to the same commands from python.

AttributeError: ‘module’ object has no attribute ‘check_output’. You need Python 2.7 or newer to use check_output(). There are other, more verbose ways of doing the same thing in older python. At least Ubuntu 12.04 has new enough python.

Error: no display specified. You can run the same commands that would work in a shell. If you can’t open firefox inside screen or over ssh in normal shell, you can’t do it from python either.

Administrivia

Tried all examples? Well done, you can now run any shell command from python.

Tested with Ubuntu 12.04 Desktop, Python 2.7.2-9ubuntu6 (2.7.3rc2).

Posted in Uncategorized | Tagged , , , , , | Leave a comment

Design a Learning Product – 5 Day Workshop in Bikent, Ankara

Innovate a new learning product in a 5 day workshop. Work in a small, diverse group of students from (Haaga-Helia) Finland and Bilkent (Turkey).

Design a Learning Product -workshop is arranged in Bilkent, Ankara. The workshop starts on w23 Sunday 2012-06-10 and ends w24 Saturday w2012-06-16.

Groups of 3-4 persons innovate a product related to learning. Each group has students from Haaga-Helia Finland and Bilkent Turkey. Also, groups will mix business and IT students.

Free Flights and Accommodation

Haaga-Helia offers free flights to participants. Bilkent University offers free simple accommodation in two-person rooms.

Who Can Apply

You have to be a student in the Haaga-Helia innovator track and be able to speak English (you’re already reading this, right?).

We can take 6-8 students. If needed, the applicants are priorized with the following: published projects (including homepages, school projects…), experience with open source technologies or success in studies and an interview.

Email You Application by Friday 11 May

Send your application by email to Tero Karvinen, firstname.lastname@haaga-helia.fi. Applications are shared with organizers in Bilkent university. In plain text email body, write

  • a short application
  • link to your homepage (or similar)
  • resume in English

Bilkent students can contact Nur Sağlam (nsaglam@) to apply.

Agenda

w23 Sun 2012-06-10	Flight to Turkey
w24 Mon 2012-06-11	1. Ideas
w24 Tue 2012-06-12      2. Design
w24 Wed 2012-06-13      3. Demonstration
w24 Thu 2012-06-14      4. Publish
w24 Fri 2012-06-15	5. Presentations & Party
w24 Sat 2012-06-16	Flight back to Finland

Flight dates have been decided. There might be other changes in daily program, though.

Apply now, email to Tero Karvinen.

Posted in Uncategorized | Tagged , , , , , | 1 Comment

Ubuntu 12.04 LTS Published – Download Here

Xubuntu 12.04 LTS released, will be supported until 2017 on the desktop. Download Xubuntu 12.04 LTS.

Final, production release. Supported for five years on the desktop.

Vanilla from Ubuntu homepage (2012-04-26: available sproradically)

Xubuntu will be published later today (according to @XubuntuLinux). Is this the final Xubuntu 12.04 (torrent, direct download)?

While you’re downloading, read my experiences with beta and release notes.

Updated link to Xubuntu 12.04 download, even before Xubuntu.com official homepage and @XubuntuLinux.

Posted in Uncategorized | Tagged , , , , , , | Leave a comment

Xubuntu 12.04 LTS on Vappu

Update: Final release is published.

I’ve been using, testing and teaching the beta version for a while. Check links for tutorials of Puppet, Vagrant and other new tools.

New Packages & Versions

Fixes to Problems

  • Gnome 2 is gone, Unity doesn’t feel ready for productive work yet. Unity is the default user interface with the hiding bar on the left. Fix: use XFCE by installing from Xubuntu CD (link below). Medibuntu switched to XFCE, so did Linux Torvalds.
  • Only <4 GB RAM with Xubuntu 32-bit. I already got used to the default PAE (physical address extension) kernel, so that 32 bit Linux can see 8 GB of memory. Possible fix (untested): sudo apt-get install linux-image-generic-pae. Obviously, if you install 64-bit version you can see all your RAM.
  • Arduino requires logging on and off before use. Fix: install instead of using a live CD.

Download

Release version of Ubuntu 12.04 LTS is out, download here.

Understand that this is Beta version, don’t install on production systems.

It’s an ISO image, so it must be burned in a special way. Ubuntu: ‘cdrecord foo.iso’ or right click and ‘Write to Disc’. Mac OSX: drag to Application: Utilities: Disk Utility. Windows: Right click and “Burn disc image” or use Infra Recorder “Burn image”.

Download xubuntu-12.04-beta2-desktop-i386.iso beta2 (700 MB iso direct) or download torrent.

Release version will probably be on cdimage by Vappu 2012-05-01: Xubuntu and vanilla Ubuntu. It will wait for you when you return from Ullanlinnanmäki…

Updated.

https://help.ubuntu.com/community/BurningIsoHowto#Mac_OS_X
Posted in Uncategorized | Tagged , , , , , , | 2 Comments

Hello Puppet – on Ubuntu 12.04 LTS

Learn “Hello Puppet World” in 5 minutes. Meet the configuration management system used by Google and Wikipedia.

Prequisites: command line, sudo and apt.

IIRQ: I wrote this article from memory. It’s based on my tests on Ubuntu 12.04 LTS beta1 and 12.04 LTS beta2.

Install Puppet Agent

$ sudo apt-get update
$ sudo apt-get -y install puppet

Create Site Manifest (the Program)

$ mkdir puppet
$ cd puppet
$ mkdir manifests
$ nano manifests/site.pp
class helloworld {
        file { '/tmp/helloPuppet':
                content => "See you at TeroKarvinen.com! "
        }
}

Run Puppet

$ puppet   --modulepath modules/   apply manifests/site.pp

First time, it shows you a log how it created the file. The second time, it doesn’t do anything – all is well already.

Verify Your Success

$ ls /tmp/helloPuppet
/tmp/helloPuppet
$ cat /tmp/helloPuppet
See you at TeroKarvinen.com!

Congratulations!

Next, you can have a look at  PuppetMaster on Ubuntu 12.04 and Puppet Reading List. Have fun with your puppet!

Posted in Uncategorized | Tagged , , , , , , | Leave a comment

OpenLayers Track Example

Show track line with OpenLayers. Click a marker to show and hide track. Live Demo.

Prequisites: Getting Started with OpenLayers and OpenStreetmap

This example demostrates:

  • Displaying tracks (routes)
  • Reading GPX-track files
  • Changing layers on top using Z-index
  • Click events

Live demo with source: Show and Hide OpenLayers GPX Track

Posted in Uncategorized | Tagged , , , , , , , , , | Leave a comment

Getting Started with OpenLayers and OpenStreetmap

Add a free, interactive map on your homepage. Just plain HTML5 and JavaScript, no special server needed. Live demo with source.

No API keys needed. And if you really like it, you can later download planet vector data for free!

Prequisites: HTML5, JavaScript

Source Code Explained

<!doctype html>

The new HTML5 Doctype. Some older examples won’t work with HTML5. Especially, you must make sure that #map div is larger than 0×0.

<meta charset="utf-8" />

Always define the charset in document. UTF-8 is the new black.

<div id="map" style="top: 0; left: 0; bottom: 0; right: 0; position: fixed;">

Fill parent with just CSS. This is the preferred method.

<script src="http://openlayers.org/api/OpenLayers.js"></script>

For good performance, JavaScript should be last, just before /body. OpenLayers.js should be called before your own code using it. To speed things up, you could also copy OpenLayers.js to your server.

map = new OpenLayers.Map("map");

OpenLayers.Map(divName) is the main container for everything. It includes layers, all the vectors and point of interest (POI) markers.

map.addLayer(new OpenLayers.Layer.OSM());

OpenLayers.Layer.OSM() is the standard OpenStreetMap Mapnik layer. This is what you normally want.

var lonLat = new OpenLayers.LonLat(24.9342, 60.2017)
	.transform(
	   new OpenLayers.Projection("EPSG:4326"), // from WGS 1984
	   map.getProjectionObject() // to Spherical Mercator
	);

You can get coordinates from OpenStreetMap.org regular web interface. Just navigate to your point of interest, using search if needed. Then click “Permalink” on bottom right. The URL shows latitude and longitude. Be careful not to mix lat and lon: they have different order in the URL and in OpenLayers.LonLat() constructor.

map.setCenter (lonLat, 14);

Center the map on the point you created. Zoom to 14, about 10 meters per pixel or 1:35.000.

Live demo: OpenLayers, OpenStreetMap, valid HTML5.

1:35.000
Posted in Uncategorized | Tagged , , , , , , , , | Leave a comment

FramedCloud PopUp with TextFile – OpenLayers

FramedCloud popups, even when data points are read from text file.

More specifically, use OpenLayers.Popup.FramedCloud with OpenLayers.Format.Text.

Live demo with full source.

Prequisites: OpenLayers basics, Javascript, OOP, reading API descriptions.

OpenLayers is a javascript library for creating interactive maps. It’s useful with OpenStreetMap.org, the worlds largest free map.

Data Points from Text File

Consider a TSV file, like this one from the examples

lat	lon	title	description	icon	iconSize	iconOffset
48.9459301	9.6075669	Title One	Description one<br>Second line.<br><br>(click again to close)	Ol_icon_blue_example.png	24,24	0,-24
48.9899851	9.5382032	Title Two	Description two.	Ol_icon_red_example.png	16,16	-8,-8

We can read it with OpenLayers.Format.Text. Because of Protocol.HTTP, this only works on the server, eg. “http://”, not from disk “file:///”

protocol: new OpenLayers.Protocol.HTTP({
        url: "textfile.txt",
        format: new OpenLayers.Format.Text(
        {
                extractStyles: true,
                extractAttributes: true
        })
})

Cool Popups with FramedCloud

When displaying, you can use FramedCloud, thus creating pretty popups

popup = new OpenLayers.Popup.FramedCloud("chicken",
        feature.geometry.getBounds().getCenterLonLat(),
        new OpenLayers.Size(100,100),
        content,
        null, true, onPopupClose);
feature.popup = popup;
map.addPopup(popup);

Enjoy Live Demo

Live demo with full source.

Posted in Uncategorized | Tagged | 2 Comments

PuppetMaster on Ubuntu 12.04

Control a whole network of computers from PuppetMaster.

Wikipedia uses puppet for servers, Google uses puppet for OSX laptops. With this article, you can use puppet with Ubuntu servers and desktops.

Prequisites

To follow this article, you should be fluent with command line, apt, sudo, installing and configuring daemons. You should have basic knowledge about public key encryption, client server -model and networking.

I assume you do the obvious stuff without mentioning: Setup and use SSH to connect to your hosts, as logging in and out of hosts is not listed here. Run ‘sudo apt-get update’ before other apt commands.

I tested with Ubuntu 12.04 LTS. If you want to use 10.04 LTS, you can find the new Puppet 2.7.x packages in backports. At least 20 persons have successfully installed PuppetMaster with this guide.

Conventions

We have two computers, hostnames master and slave. Prompt shows where the commands are given, eg. running ‘pwd’ on host master.

master$ pwd

Test connectivity

slave$ ping -c 1 master.local

If your master doesn’t answer, fix that first.

Feel free to use whatever hostnames you want. If you hosts are in the same local network, you can use Avahi/ZeroConf/Rendevouz to use hostname.local names. If your master has a public DNS name or a dynamic name, you can use that, too.

To use .local names, you might need to ‘sudo apt-get -y install avahi-utils’.

Install PuppetMaster

master$ sudo apt-get -y install puppetmaster

Regenerate Master Certificate

Let’s create PuppetMaster certificate with correct names. Clients will only accept certificate if it matches the DNS name they use for contacting the master.

To avoid using hackish “puppet in hosts” method, create a certificate with all names of master. First, remove the old certificate

master$ sudo service puppetmaster stop
master$ sudo trash /var/lib/puppet/ssl

Add master’s name to config

master$ sudoedit /etc/puppet/puppet.conf

add these names under [master] heading

dns_alt_names = puppet, master.local, puppet.terokarvinen.com

Certificate is automatically generated when you start PuppetMaster

master$ sudo service puppetmaster start

You can verify certificate details with ‘sudo ls /var/lib/puppet/ssl/certs/’ and ‘sudo openssl x509 -in /var/lib/puppet/ssl/certs/puppet.terokarvinen.com.pem  -text|grep -i dns’. It should show all of your DNS names.

Connect from Slave

slave$ sudo apt-get -y install puppet
slave$ sudoedit /etc/puppet/puppet.conf

Add master DNS name under [agent] heading. Puppet will connect to server.

[agent]
server = master.local

Allow puppet slave (aka agent) to start

slave$ sudoedit /etc/default/puppet

Change to yes:

START=yes

If you have connected to master before, force slave certificate regeneration with ‘sudo service puppet stop’ and ‘sudo trash /var/lib/puppet/ssl’. The new slave certificate will be generated the next time puppet starts.

Start puppet agent

slave$ sudo service puppet restart

Slave should now connect to master.

Sign Slave Certificate on Master

master$ sudo puppet cert --list
master$ sudo puppet cert --sign slave.example.com

Of course, use the actual name of the slave you have chosen.

Create Site Manifest and a Module

Create site manifest, the config file that includes everyghing else:

master$ cd /etc/puppet
master$ sudo mkdir -p manifests/ modules/helloworld/manifests/
master$ sudoedit manifests/site.pp

Add just one line

include helloworld

Create a hello world module

master$ sudoedit modules/helloworld/manifests/init.pp

Write the module

class helloworld {
        file { '/tmp/helloFromMaster':
                content => "See you at http://terokarvinen.com/tag/puppet\n"
        }
}

Test & Enjoy

Puppet will automatically fetch configuration every now and then. Restarting (or usually, just reloading) the service will fetch and apply configuration immediately.

slave$ sudo service puppet restart

Congratulate yourself when you see the file from PuppetMaster

slave$ cat /tmp/helloFromMaster
See you at http://terokarvinen.com/tag/puppet

Administrivia

Tested on two instances of Xubuntu 12.04 LTS beta1 on Vagrant running on Xubuntu 12.04 LTS beta2. Also, at least 20 persons have successfully installed PuppetMaster with this tutorial.

Update: Fixed puppet.conf path on slave. About 20 persons successfully tested this guide.

Posted in Uncategorized | Tagged , , , , , | 3 Comments

Puppet Reading List

Start with the material marked with an asterisk “*” and bold.

Getting Started with Puppet

Hello Puppet – on Ubuntu 12.04 LTS*

Learning Puppet*

Master and Slave

PuppetMaster on Ubuntu 12.04*

Modules

Learning Puppet: Modules and Classes (Part One)

Learning Puppet: Parameterized Classes (Modules, Part Two)

Puppet Module Cheat Sheet

Puppet 2.7 Reference: Module Fundamentals

Sample modules

Ever wondered how the Wikimedia servers are configured?

Puppet Forge

Search “puppet” on GitHub

Some modules by students in timetable comments

Language

Type Reference

Puppet Cookbook*

Language: Advanced techniques

Custom functions

Updated.

Posted in Uncategorized | Tagged | Leave a comment
  • Picks

  • Boxing Clock for AndroidOcton8 Diving T-ShirtsShaking Tower Panda Android GameLearn Chinese with Android
  • Student projects