Archive for category Shell Scripting
Adding the ARDX.ORG source code to your Arduino Example folder
Posted by grymoire in Hacking, Shell Scripting, Technology on October 16, 2011
I purchased the ARDX kit from Lady Ada, and I wanted to experiment with all of the source code from the ARDX web site. However typing in the link and copying the source code didn’t automatically add it as an example. Instead I had to re-save it as a sketch. So instead, I wrote a shell script called ARDX.sh
This does several things.
- It downloads all of the sketches.
- It creates a directory to store all of the sketches
- It creates a directory for each one of the sketches.
- It renames the source code into a *.pde file
- It moves the *.pde file into the proper sketch folder
- It removes all files created during the process. that are no longer needed.
To use the script, simply type
./ARDX.sh
# This creates a folder called "Ardx"
mv Ardx .../arduino-0022/examples/.
Here is the script. Note how I used the $debug variable. This let me select if I wanted to show the commands or execute the commands.
#!/bin/sh # Remove the '#' at the beginning of the next line to debug this script #debug=echo if [ ! -d Ardx ] then mkdir Ardx fi cd Ardx Examples="01 02 03 04 05 06 07 08 09 10 11 12A 13A" for i in $Examples do $debug wget -r http://ardx.org/CODE$i done $debug wget -r http://www.ardx.org/src/circ/CIRC12-code-ADAF.txt $debug wget -r http://www.ardx.org/src/circ/CIRC13-code-ADAF.txt # Now create a directory for each of the examples Examples="01 02 03 04 05 06 07 08 09 10 11" for i in $Examples do if [ ! -f ardx.org/src/circ/CIRC$i-code.txt ] then echo unable to find file ardx.org/src/circ/CIRC$i-code.txt else $debug mkdir CIRC$i $debug mv ardx.org/src/circ/CIRC$i-code.txt CIRC$i/CIRC$i.pde fi done # Now remove the old files in $debug mkdir CIRC12A $debug mv www.ardx.org/src/circ/CIRC12-code-ADAF.txt CIRC12A/CIRC12A.pde $debug mkdir CIRC13A $debug mv www.ardx.org/src/circ/CIRC13-code-ADAF.txt CIRC13A/CIRC13A.pde $debug /bin/rm -r ardx.org www.ardx.org
Restart your Arduino session, and when you look in examples, you will see a new folder called Ardx. Inside is a sketch for each of the examples.
Cracking Alchemy on the iPhone
Posted by grymoire in Hacking, Shell Scripting on March 13, 2011
Someone asked me how I got all of the Recipes for Alchemy on the IPhone. The answer – I cheated. I hacked the system.
Let me describe how I did this. There is some simple perl scripting involved, which might be useful to people.
Step 1 – Jailbreak the iPhone
I’m a hacker. What can I say.
Step 2 – Getting the Alchemy Files
Find where the Alchemy application is stored.
You can open a terminal window, run find, and look for some file that has Alchemy in the name.
find / -name Alchemy*
Instead of a terminal window, you might find it easier to ssh into the iPhone. You do have to use Cydia to install the various extra programs. I had the following installed
- Bourne-Again shell
- Openssh
Once OpenSSH is installed, you have to use your iPhone, check the settings, and find your IP address. Say it’s 192.168.1.222. The iPhone has a habit of shutting down its IP address if not in use. So refresh the IP address using the DHCP refresh, and then type
ssh -l mobile 192.168.1.222
The password for the user “mobile” is “alpine”
Then use ‘find’ to locate the file. Once you have found this, grab all of the Alchemy files by typing the following on your Linux (ofr Windows with Cygwin)
Copying the Alchemy files onto your Linux machine
scp -r mobile@192.168.1.181:/var/mobile/Applications/9E1E4FC0-B6CE-4036-8DFB-BCB5666D4741/Alchemy.app .
This will copy all of the files onto your computer.
The string above is on my machine. Your mileage may vary.
You can examine the files to see what they contain using “string” “od”, “emacs”, or whatever. It turns out that the file you want is Sparing.plist.
Next you want to decode the resource file that contains the recipes. I use perl. Looking at the various packages, the one that makes the most sense is the Mac::PropertyList module. Looking at the Dependency, it requires the XML::Entities module. Therefore, after you install perl, download and install the two Perl modules.
Step 3 – Getting Ready to use perl
Making it easier to install perl modules
I prefer to make sure /usr/local belongs to a certain UNIX group(5), such as adm, and that all of the subdirectories have group write permission.
In other words, type the following commands as root
chgrp -R adm /usr/local chmod -r g+w /usr/local
Installing the two perl Modules
This way you can install modules without requiring root access. To make the modules, after you download them, type
tar xfz XML-Entities* cd XML-Entities perl Makefile.pl make make install cd .. # And now do the next one tar xfz Mac-Properties* cd Mac-Properties* perl Makefile.pl make make install cd ..
Step 4-Extracting the Alchemy data from the Resource file
The Mac::PropertyList module creates a complex data structure with all of the information. The elegant thing to do is to write one perl program that reads the data and prints the results you want to get. But that takes more work. I prefer the “get the job done as easy as possible” school of programming. Rather than try to figure out the format of the complex data, the simplest thing to to is to let perl decode the binary information for you. The program to use is Data::Dumper which outputs the structure of complex data. This is essential for the Perl programmer. You just give it a pointer to a complex piece of data, and Data::Dumper will describe it. The code to read the file and dump the results is below:
#!/usr/bin/perl my $filename="./Alchemy.app/Sparing.plist"; use Mac::PropertyList qw( :all ); use Data::Dumper; my $data = parse_plist_file( $filename ); my $text = plist_as_string( $data ); print Dumper($data);
Call this program List, add the +x attribute with chmod, and they type
./List >List.out
If you look at the output of this file, you will see something that looks like this
VAR1 = bless( {
'Metal+Electricity' => bless( do{(my $o = 'Aluminium')}, 'Mac::PropertyList::string' ),
'Oil+Tool' => bless( do{(my $o = 'Petrol')}, 'Mac::PropertyList::string' ),
etc.
Aha! Clearly, Metal+Electricity creates Aluminum. Let’s write a perl script that reads this file, and outputs the recipe.
But some of the lines, like the first one, is a different format. Well, perl can handle this very easily.
Parsing ASCII in perl
I usually use the following template to do string parsing
#!/usr/bin/perl -w
use strict;
#my filename="List.out";
my $line;
while (defined($line=<>)) {
# 'Metal+Electricity' => bless( do{(my $o = 'Aluminium')}, 'Mac::PropertyList::string' ),
if ($line =~ /complexstring/) {
} elsif ($line =~ /string/){
printf("You didn't match this line: $linen");
} else {
printf("Can't parse line: $linen");
}
}
I include the string I am trying to parse as a comment, to help me get the regular expression correct.
The second string is how I debug the first string
First attempt at matching an ASCII line in perl. I replace the strings I am tring to match ‘Metal+Electricity’ and ‘Aluminum’ with “.*”
if ($line =~ /'.*' => bless( do{(my $o = '.*')}, 'Mac::PropertyList::string'/) {
} elsif ($line =~ /bless/){
But I want to remember the strings found within the ‘…’, so I need to add parenthesis around them, so perl will remember them. This would be
if ($line =~ /'(.*)' => bless( do{(my $o = '(.*)')}, 'Mac::PropertyList::string'/) {
printf("$1 => $2n");
} elsif ($line =~ /bless/){
This is a start, but there is a problem. When I use sed, I need to put backslashes before the parenthesis to mark them as special. Perl is the opposite. A backslash means the character is NOT special, or NOT a metacharacter.
So I need to put backslashes around the other parenthesis
if ($line =~ /'(.*)' => bless( do{\(my $o = '(.*)')}, 'Mac::PropertyList::string'/){
I also needed to put a backslash before the backslash. Running this gives me the error
Global symbol "$o" requires explicit package name at ./ParseBug.pl line 6.
(Smack forehead) I also need to put before the ‘$’
Sometimes I need to experiment with the regex, and the second line, where the ‘/bless/ is show, is a guess. Note that this also prints out lines that don;t match anything.
Eventually, I get this right. The final version is
-
#!/usr/bin/perl -w
use strict;
#my filename="List.out";
my $line;
while (defined($line=<>)) {
if ($line =~ /'(.*)' => bless( do{\(my $o = '(.*)')}, 'Mac::PropertyList::string'/){
printf("%s=%sn", $2, $1);
} elsif ($line =~ /bless/) {
printf("missed line: $linen");
} elsif ($line =~ /VAR1/) {
} elsif ($line =~ /Mac::PropertyList::dict/) {
}else {
printf("Can't parse line: $linen");
}
}
This prints out all of the recipes
sed and the t character
Posted by grymoire in Shell Scripting on July 18, 2010
Dear Grymoire,
I am trying to convert a pipe character into a tab. I tried
sed y/|/t/ <in>out
but it did not work. Instead of a tab, I got a ‘t’. What did I do wrong?
I thought this was a good lesson, as there are several misunderstandings a beginner might have when they write a shell script.
The first lesson concerns meta-characters and quoting, which I covered in my tutorial.
A meta-character is simply a character that has a special meaning.
The character ‘t’ is a normal character. The character ” is a special character. It has a special meaning. However, the meaning of this character changes depending on how and where it is used. You have to know the context. ‘|’ is another example. It may or may not be a meta-character.
So what was wrong with ‘|’ and ‘t’?
To break down this problem into steps, there are some basic concepts.
In Unix shell scripts. the shell is the interface between you and the computer. When you write a shell script, and that script calls a program like sed, there are two steps
- The shell reads the shell script.
- The shell passes the characters it sees to the program.
Let’s take for example the above script. How does the shell see the script? The easiest way is to insert echo as the first command on the line.
We also have to remove the <in >out so we can see what happens. So the new command is simply
echo sed y/|/t/
And this command echos the following to the screen:
sed y/|/t/
So the pipe character is seen by the shell and echoed. But where is the tab character?. What happened?
Normally, the shell sees the ‘|’ character as a special character. It’s a meta-character because of this special meaning. It tells the shell to start a new process, and you take the output of the previous character, and send it to the new process.
Remember that the shell has three kinds of quotations, a single quote character, a double quote character, and a backslash.
So when you quote a special character,it becomes a special kind of special character.
In other words, it’s normal. Crazy rules, but that’s how it works. Quoting special characters in the shell make them normal and ordinary – just like the letters a-z, etc.
So the ‘|’ is correct. So is using single or double quotes. So why didn’t ‘t’ work? Well, on some systems, sed will output either a ‘t’ or a ‘t’ instead of a tab. What is happening?
The second lesson is this:
To understand what is a meta-character, you have to be familiar with the program that is looking at the character.
The shell sees ‘t’ to be the same as ” followed by ‘t’, so it treats them the same way.
What does sed do with ‘t’? Well, it depends.
Some versions of sed treat ‘t’ to be the same as ‘t’. Others treat ‘t’ as a tab.
The person who send me this problem was trying to get a tab character into this part of the script. Well, there are two approaches:
- Let sed convert ‘t’ into a tab
- Let the shell convert ‘t’ into a tab, and pass this into sed.
This causes a problem if neither the shell nor sed interpret ‘t’ to be a tab character. There are some programs that do treat ‘t’ is a tab character. These include
- The C programming language in a string
- The print(1) command, which is based on the C language
- perl
- awk
- The tr(1) command
- Some versions of the echo(1) command.
- … and several other computer languages.
But the shell does not. And sed may (or may not). So what’s the answer?
It’s quite simple.
If you want sed to insert a tab, you may have to include the tab character in the script.
In other words, the answer is to use the following
sed ‘y/|/ /’ <in >out
Where the character between the ‘/…/’ is a tab character. Unfortunately, I cannot show a tab character there, because it looks like zero or more spaces, depending upon the tab stops in your terminal. However, you are reading this using a browser, which cannot show tabs either.
So how do you get a tab character in the script? If you are using an editor, you can edit a file that contains the script and insert a tab character.
This seemingly simple task can be tricky. Some editors convert tabs to spaces. This can depend upon the settings in your Unix terminal.
To make sure you have a tab character in the script, use ‘od -c’ to display the file. In other words, if you had the script
#!/bin/sh sed 'y/|/ /'
and typed in od -c script, you would see
0000000 # ! / b i n / s h n s e d ' y 0000020 / | / t / ' n 0000027
If this doesn’t work, some of your terminal settings may be wrong.
A second way to get it to work is to enter the script from the command line. The Control-V character tells the terminal handler to “quote” the next character. If it is converting tabs to spaces. Control-V <Tab> may let you type a real tab character.
The third way is to use another program to create the tab character. tr(1) understands ‘t’ so let it create it for you. One way is to set a variable to be equal to a tab character, and then use this variable inside of double quotes:
#!/bin/sh tab=`echo a | tr a 't'` sed "y/|/$tab/" <in >out
This format is useful if you want to publish a script on the web. That’s because the script does not contain any non-printing characters. This makes it easy to cut and paste a script. Copying a tab character, and pasting it into another window may not work.
There is one more point
Don’t change characters when you don’t need to
In other words, instead of changing the pipe character to a tab, just use the pipe character as the field separator. Awk allows this
Just use
awk -F’|’ ‘{print $3}’
or whatever.