Super Spread Sheet S³

Or little computing tricks and hacks

Category Archives: Command line

Adventures with Ubuntu in a MacBookPro9,2

My daughter came to me one day: “Mum, my Mac is kaput”. When upgrading, the computer just hang. After all the diagnosis possible, we figured it was a disc crash.

A mac techie acquaintance took the computer and after checking it up told us that he could replace the disc and … install Ubuntu! Yes please!

After getting the computer back I started playing with it, connect to the wifi, install this, download that, why is the Ubuntu version only 14.04? Let’s upgrade. After all we are only a few days from the next release 15.10. Hmm, there are errors. We need to reboot, yes, no, ahhh. Kernel panic…

… After some research I had some interesting findings.

There seems to be a tie between the Mac model and the Ubuntu release. This page shows the recommended Ubuntu release to the specific MacBookPro hardware model. They recommend the latest LTS when the user is not sure of the release to install. I was reticent to leave 14.04LTS, but looking at this wikipedia page, I was reassured that this particular version’s support runs until 2019-04! By then this Mac should be history!

To install according to the Mac’s model, first find out the hardware type by typing the following:

sudo dmidecode -s system-product-name

The output in my case:

MacBookPro9,2

And there is where I noticed that 15.04 was not going to work. So I proceeded to reinstall 14.04LTS from a usb stick and that was like a breeze, only after reading how to boot from a usb stick in a Mac:

Insert the Ubuntu LiveCD into your Mac and Shutdown. Restart the Mac and hold the Option Key. When the boot selector screen comes up, choose to boot from the CD.

The full installations instructions can be found here, but I just followed the section “Single-Boot: Ubuntu Only”.

All good except that the wireless card did not seem to be set up. But it was working before so it can be done. I did get scared when I clicked on the MacBookPro9-2/Utopic Unicorn link, and it read that wireless was not supported. But Utopic Unicorn is 14.10. And I have 14.04 Trusty Tahr.

Roughly these are the steps to follow to set up the wireless connection.

Identify the wireless chipset

This can be done in a couple of ways:

  • lspci | grep Network
  • lspci -vvnn | grep -A 9 Network

From the commands I learned that

  • The Chip ID is BCM4331,
  • The PCI-ID is 14e4:4331, and
  • Kernel driver in use is bcma-pci-bridge

Find the drivers for the chipset

This guide contains a full description of specific drivers supporting Broadcom BCM43xx Chipset. And there are a different instructions that one could follow. IN my case the chipset was supported by more that one driver but what worked for me was the section b43 – No Internet access:

  1. Install the b43-fwcutter package.
    cd /media/pool/main/b/b43-fwcutter/
    sudo dpkg -i b43-fwcutter* 
    
  2. Download the firmware file from here unto a computer with internet connection.
  3. Copy the file to your working directory (yes, using a usb stick). In a terminal use b43-fwcutter to extract and install the firmware:
    tar xfvj broadcom-wl-5.100.138.tar.bz2
    sudo b43-fwcutter -w /lib/firmware broadcom-wl-5.100.138/linux/wl_apsta.o
    
  4. Restart the computer or reload the b43 module by switching between drivers. I did the later.
    First unload all conflicting drivers (this includes removing the driver you’re trying to install):

    sudo modprobe -r b43 bcma
    sudo modprobe -r brcmsmac bcma
    

    Then load the driver to use:

    sudo modprobe b43
    

And by magic I now have a wireless connection, and life is good again!

Related links

  1. https://en.wikipedia.org/wiki/List_of_Ubuntu_releases#Ubuntu_15.10_.28Wily_Werewolf.29
  2. https://help.ubuntu.com/community/MacBookPro
  3. https://help.ubuntu.com/community/MactelSupportTeam/AppleIntelInstallation
  4. https://help.ubuntu.com/community/WifiDocs/Driver/bcm43xx
  5. http://www.lwfinger.com/b43-firmware/broadcom-wl-5.100.138.tar.bz2
  6. http://askubuntu.com/questions/338210/broadcom-sta-wireless-driver
  7. https://en.wikipedia.org/wiki/Station_%28networking%29
  8. https://help.ubuntu.com/community/WifiDocs/WirelessCardsSupported

Restarting unicorn after boot using mina and nginx in Rails

My teammate did a wonderful job at setting up a droplet in Digital Ocean to host a Rails app. Most of the details are here.

All of the reference to directories are based on having exactly the set up described in that blog post.

However, we noticed that unicorn was not being restarted after booting, having to do so by hand. Not quite a problem if you are on developing/staging states, but in full production a real problem. Except, we could not use a simple command line, because unicorn has to be started by mina.

Looking at the config/deploy.rb file, I noticed the following:

    to :launch do
      invoke :'unicorn:restart'
    end

so the command line to start unicorn should be:

mina unicorn:restart

Actually the command only works if in the correct directory: /home/deployer/my-app-name/current as follows:

bundle exec mina unicorn:restart

So now, all I had to do was to turn that command into a script called at boot. By the way, the droplet uses Ubuntu.

After a few attempts, I created a file called /etc/init.d/my_unicorn, you have to have root privileges for this, with the following format:

#!/bin/sh -e
# upstart-job
#

echo "starting unicorn after reboot"
exec sudo -u deployer sh -c "cd /home/deployer/my-app-name/current && /home/deployer/.rbenv/shims/bundle exec mina unicorn:restart"

(Looking back “-u deployer” might not be needed but I did not test it.)

By just adding that script, I was able to run

$sudo service my_unicorn start

But, it was not being called at boot yet. I needed to add new service (i.e. my_unicorn) to startup. For that, the following command is needed:

sudo update-rc.d my_unicorn defaults

And that should have worked on reboot, but it wasn’t.

Every time mina is called, it asks for authentication, and after 3 failed attempts it quits with an error. The solution was to create a ssh key and add it to the authorized_keys file, the same way you do for sites like github.

And that is it, if I haven’t forgotten anything!

Rotate image with convert

Another simple but powerful use of convert:


convert original.jpg -rotate -90 new.jpg

convert original.jpg -rotate 90 new.jpg

To rotate the image to the left (anticlockwise) and to the right (clockwise), respectively.

Take from https://help.ubuntu.com/community/Photos/RotatingPhotos,

Physical versus core processors

I finally found a description of the difference between the physical processor and its cores.

The physical processor in the modern computer comes nowadays with two or four processor cores, respectively called dual or quad-core. These cores are like virtual processors and can handle instructions as if they were standalone.

In the system description of the computer, a single physical processor may appear as two (for dual-core) or four processors (for quad-core), as the system might describe virtual processors and not actual sockets.

To query for the number of processors and cores, in ubuntu use the following command:
cat /proc/cpuinfo
Doing a grep on “processor” will summarize the number of cores.

(The info on this post was all taken from here.)

Image to PDF

Such a simple thing to convert an image file into a PDF file. Internally what is happening is that the image file is wrapped in PDF headers:

convert infile.png outfile.pdf

And as always, any of the image extensions can be used.

Rewriting history

It is said that

those who do not remember their history are condemned to repeat it.

Well that is what happened me. My phone is very picky about the direction is which is needs to be held to catch the right angle of a film. So, more than once I was face with having to transpose the video. And more than once I used the search engines to find the answer. And had even written a post, which I could not remember.

This is why I could not remember. I did not write what did not work. Here it is:

The first search gave me this as an alternative, found here
avconv -i input.mp4 -c:v libx264 -c:a copy -vf "transpose=1" output.mp4

This command was taking forever, bloating the file, and making my computer over heat. I did not look for the reasons, as just about now, I started remembering that I had found a way…

The second try found here was
ffmpeg -i in.mov -vf "transpose=1" out.mov
Yes, we are getting closer, I remember ffmeg. So I replace .mov by mp4 and launched. This time the computer was not reheating was it was taking a long time. The file was growing slower than before. Still.

Then I remember to look in previous posts and bingo. The magic command originally from here:
ffmpeg -i input.mp4 -vcodec libx264 -preset medium -crf 24 -threads 0 -vf transpose=1 -acodec copy output-file.mkv.
Ran in a split of a second and the resulting file one fifth of the original.

Note on 24 August 2014

Last time when I tried to run ffmpeg in Ubuntu 4.14, noticed with a surprise, that ffmpeg is no longer installed in the system. After some research, in the Ubuntu forums one of the solutions given is to download the Linux build from here, extract and execute, no installation.

As an added bonus I was able to convert into the same format, not possible with the drivers I had before.

That worked for me. I did not find any standard installed way of doing it.

Peculiarities in vi

I have been using the vi editor for some time now, and is still my favourite editor. emacs is very powerful, and it has a lot of tools but if you don’t need nor use those tools, well is like having a complicated gedit.

I really had a laugh when I started the tutorial “Learn Python the Hard Way”, and was told not to use vi because it is too complicated. If you are not good at the keyboard, as in you are far from touch-typing, it will take you a while to get around to it. But when you do, you can really fly. There is also gvim, a combination of GUI and vi.

Nevertheless, vi has some quirks that make me wonder…

Starting one of my tutorials (Ruby on Rails?), it suggested to the reader to adjust the settings of the editor to have some automatic indentation for nested blocks. In vi this was:

set expandtab
set shiftwidth=4
set ts=4
set autoindent

And this works wonderfully when coding. But when doing cut-and-paste, the results leave much to be desired. So each time I want to cut and paste I had to unset the last line. Initially I did it by commenting the line, that is adding a double-quote at the beginning of the line. But then I decided to do it within the editor.

My first guess was:

:unset autoindent

Makes sense right? Except that is not… The correct command is:

:set noautoindent

How unconventional is that!

Video 90° rotation

During the my daughter’s music camp, I recoded some videos rotated 90°. VLC offers an option for rotating but just for viewing. Instructions here.

To permanently rotating the video use the powerful ffmpeg:

ffmpeg -i input-file -vcodec libx264 -preset medium -crf 24 -threads 0 -vf transpose=1 -acodec copy output-file.mkv

Taken from the Ubuntu Forums.

The output is in the Matroska Multimedia Container file format (.mkv) and you can find an explanation in Wikipedia.

Git 2.0

The behaviour of the version 2.0 of git will changed for git push/pull.

Before, the default was matching:

matching – push all branches having the same name in both
ends. This is for those who prepare all the branches into a
publishable shape and then push them out with a single command.
It is not appropriate for pushing into a repository shared by
multiple users, since locally stalled branches will attempt a
non-fast forward push if other users updated the branch.

After 2.0, the new default is simple:

simple – like upstream, but refuses to push if the upstream
branch’s name is different from the local one. This is the
safest option and is well-suited for beginners.

Before but close to version 2.0. the user will get a message about this issue everytime that git pull/push is ran:

$ git push
warning: push.default is unset; its implicit value is changing in
Git 2.0 from 'matching' to 'simple'. To squelch this message
and maintain the current behavior after the default changes, use:

  git config --global push.default matching

To squelch this message and adopt the new behavior now, use:

  git config --global push.default simple

See 'git help config' and search for 'push.default' for further information.
(the 'simple' mode was introduced in Git 1.7.11. Use the similar mode
'current' instead of 'simple' if you sometimes use older versions of Git)

Everything up-to-date

So as the message states and for the sake of repetition, to keep the pre 2.0 behaviour use:
git config --global push.default matching

To adopt the new behaviour use:
git config --global push.default simple

The latter is recommended when the repository is shared and for beginners.

Multiple mailboxes with mutt

The previous blog entry explained in detail how to install and set up mutt to retrieve and send mail using a server such as gmail. Here I’ll explain how to manage several mailbox using the same set up.

For explanations on the configuration files check the previous post.

To fetch from several mailboxes, the file .fetchmailrc needs extra information. For all the common information to all servers, start the file with the key word defaults and then include specific information under each poll:

defaults
proto POP3
mda "/usr/bin/procmail -d %T"
options
no keep
ssl
sslcertck
sslcertpath /etc/ssl/certs

# First gmail account, the official
poll pop.gmail.com
user "user1@gmail.com"
there with password "password1"
is "user1" here

poll pop.gmail.com
user "user2@gmail.com"
there with password "password2"
is "user2" here

All of the servers will use POP3 and the same SSL certificates. The MDA is procmail. Notice that any server can be used. I just haven't tested any other so I don't have the server URL. Fetchmail man pages suggest to stash the account/password pairs in your $HOME/.netrc file where this information can be used for FTP and other programs. Check that this is working by typing

  fetchmail -v

If you have different mailboxes, they are probably of a different type, so you want to sort them in different inboxes. Procmail will do this through the recipes in the .procmailrc file:

# Environment variable assignments
PATH=/bin:/usr/bin:/usr/local/bin 
VERBOSE=off                   # Turn on for verbose log
MAILDIR=$HOME/Mail            # Where procmail delivers recipes
LOGFILE=$HOME/.procmaillog    # Keep a log for troubleshooting.

# Recipes
:0:
* ^To.*user1|^From.*user1|user1@gmail.com
mailbox1

:0:
* ^To.*user2|^From.*user2|user2@gmail.com
mailbox2

In the first recipe all mail sent to or received from, actually all mail which includes the full email user1@gmail.com in the header is to be directed to mailbox1. Similarly for for the next recipe. This will make sure that even the Cc are included.

Now mutt needs to know about the different mailboxes and about the different accounts available to send mail. The file .muttrc needs to include the following:

  mailboxes ! +mailbox1 +mailbox2

After running fetchmail, procmail has done the sorting and running

  mutt -y

will show the list of all the mailboxes with N next to the ones with new mail. I included the following line in the configuration file for a cleaner view of the listing:

  set folder_format="%2C %t %N %8s %d %f"

To be able to send mail from these different mailboxes, both mutt and msmtp have to have the right configurations files. For mutt, the easiest thing is to have as many .muttrc files as email address you want to send mail from, changing the appropriate details for setting msmtp in mutt, namely:

set realname = "John The Big Brother"
set from = john.example@gmail.com

and name each of them differently, say .muttrc_mb1 and .muttrc_mb2. Then create an aliases, in .bashrc like so (taken from this link):

alias mb1='mutt -F /home/you/.muttrc_mb1'
alias mb2='mutt -F /home/you/.muttrc_mb2'

which meant that you have to know which email address you want to send email from before calling mutt. What I do is to have a default and the others are for specific occasions.

You can also try muttprofile, but I have not bothered to try it as the current set up does not bother me.

The .msmtprc describes each email account under the keyword account. At the beginning of the file under defaults, you can set the global values for all the accounts. The define each account in turn, with the option of later ones inheriting from earlier ones.

# Set default values for all following accounts
defaults
tls on
tls_starttls on
tls_trust_file /etc/ssl/certs/ca-certificates.crt
logfile ~/.msmtp.log
auth on

# First account
account main
host smtp.gmail.com
port 587            
from "john.example@gmail.com"                 
user "john.example"       
password "rover"

# Second account
account other : main
host smtp.gmail.com
port 587            
from "john.other@gmail.com"                 
user "john.other"       
password "rover2"

# Set a default account
account default : main