BASH Scripting - Revision history

Cclark at 15:31, 19 February 2013

2013-02-19T15:31:12Z

Cclark: moved Scripting to BASH Scripting: page will cover only bash shell scripting

2013-02-19T15:25:58Z

moved Scripting to BASH Scripting: page will cover only bash shell scripting

Cclark: Replaced content with "`bash` supports many features available in traditional programming languages. However, since `bash` is a shell first, and programming language second, t…"

2013-02-19T15:22:58Z

Replaced content with "<code>bash</code> supports many features available in traditional programming languages. However, since <code>bash</code> is a shell first, and programming language second, t…"

Cclark: /* Text Manipulation Commands */

2011-09-17T17:44:36Z

‎Text Manipulation Commands

Cclark at 17:44, 17 September 2011

2011-09-17T17:44:05Z

Cclark: /* Pipes (the case for stdin) */

2011-09-07T01:39:30Z

‎Pipes (the case for stdin)

Cclark: /* Pipes (the case for stdin) */

2011-09-07T01:30:01Z

‎Pipes (the case for stdin)

Cclark: /* Pipes (the case for stdin) */

2011-09-06T21:46:56Z

‎Pipes (the case for stdin)

Cclark: /* Pipes (the case for stdin) */

2011-09-06T21:42:37Z

‎Pipes (the case for stdin)

Cclark at 15:07, 6 September 2011

2011-09-06T15:07:19Z

← Older revision		Revision as of 15:31, 19 February 2013
Line 1:		Line 1:
	<code>bash</code> supports many features available in traditional programming languages. However, since <code>bash</code> is a shell first, and programming language second, there are a few quirks that you need to be aware of. This page covers several topics that are useful when writing bash scripts and provides examples.		<code>bash</code> supports many features available in traditional programming languages. However, since <code>bash</code> is a shell first, and programming language second, there are a few quirks that you need to be aware of. This page covers several topics that are useful when writing bash scripts and provides examples.
		+
		+
		+	====variables====
		+	Bash supports several "programming" concepts, including variables. However, the syntax for using variables in bash can seem a little strange. When setting variables, use only the variable name
		+	and make sure there is not a space between the variable name and the equal sign. When deferencing a variable, put the variable name in <code>${}</code>. Variable substitution will be performed inside of double
		+	quotes, but not inside of single quotes. Here is an example:
		+	<code>
		+	#! /bin/bash
		+
		+	msg="hello world"
		+
		+	echo "i am supposed to give you this message: ${msg}" # will print: i am supposed to give you this message: hello world
		+	echo 'i am supposed to give you this message: ${msg}' # will print: i am supposed to give you this message: ${msg}
		+	echo "i am supposed to give you this message: '${msg}'" # will print: i am supposed to give you this message: 'hello world'
		+	echo "i am supposed to give you this message: \"${msg}\"" # will print" i am supposed to give you this message: "hello world"
		+
		+	The shell automatically sets several special variables. These special variables <code>${1}, ${2}, ...<code> contain the arguments that were given to your script.
		+	<code>
		+	#! /bin/bash
		+
		+	echo "arg 1: $1"
		+	echo "arg 2: $2"
		+	echo "arg 3: $3"
		+	echo "not sure about the rest"
		+	</code>
		+
		+	====conditionals (<code>if</code> statements)====
		+	Bash also supports running code "conditionally". This is useful for checking that the correct number of arguments where given to your script, or
		+
		+	====Other====

← Older revision		Revision as of 15:22, 19 February 2013
Line 1:		Line 1:
−	Scripting refers to the practice of writing "scripts" to perform tasks on a machine. A shell script is a file that contains a set of command that well be executed as if they were typed in at the command line. Shell scripts make use of the wide range of command line tools that already exist for processing text and data. Scripts have been used by system administrators for years to automate repetitive task, but scripting has many applications in computational physics and high performance computing.	+	<code>bash</code> supports many features available in traditional programming languages. However, since <code>bash</code> is a shell first, and programming language second, there are a few quirks that you need to be aware of. This page covers several topics that are useful when writing bash scripts and provides examples.
−
−	~~==Pipes (the case for stdin)==~~
−	~~One of the most powerful features of Unix, and therefore Linux, is pipes. Pipes direct the standard output of one program to the standard input of another program. So, for exmaple, the~~ <code>~~cat~~</code> ~~command prints the contents of a file to standard output~~
−	~~$ cat file.txt~~
−	~~1 4~~
−	~~2 3~~
−	~~3 2~~
−
−	In this example, the file named file.txt contains three lines. The <code>cat</code> command prints these lines to standard output. Now, the <code>grep</code> command reads lines from standard input, and prints lines that match a pattern to standard output. So, if we wanted to see only the lines in ~~file~~.~~txt that contain the number 3~~, ~~we can ''pipe'' the standard output of the~~ <code>~~cat~~</code> ~~command to <code>grep</code>~~
−	~~$ cat file.txt \| grep "3"~~
−	~~2 3~~
−	~~3 2~~
−
−	~~Here, we have two ''separate'' commands to build~~ a ~~''single'' output. There is no limit to the number of pipes we can chain together (or, if there is~~, ~~this limit is much larger than you will need). We could take the output of the last command~~ and ~~''pipe'' it through <code>sort</code> to show the lines in descending order on column two.~~
−	~~$ cat file.txt \| grep "3" \| sort -g -k 2~~
−	~~3 2~~
−	~~2 3~~
−	~~The -k 2 option tells sort to sort the data based on column 2. -g tells it to sort based on numerical value~~, ~~rather than text (ascii) value.~~
−
−	~~Pipes simply connect the standard output of one command to the standard input of another command. In fact, pipes~~ are completely separate from the commands, i.e. you can use pipes with programs you write. Since pipes work with stdin and stdout, you can write a program that reads from stdin and writes to stdout and use your programs with pipes. For example, consider a ~~simple C++ program~~ that ~~reads numbers from standard input and computes their average,~~
−	~~/* avg.cpp - compute average of a list of numbers */~~
−	~~#include <iostream>~~
−
−	~~int main()~~
−	{
−	~~double x, sum;~~
−	~~int N;~~
−
−	~~sum = 0;~~
−	~~N = 0;~~
−	~~while( std::cin>>x )~~
−	{
−	~~sum += x;~~
−	~~N++;~~
−	}
−
−	~~std::cout<<sum / N;~~
−	}
−
−	If you ~~just run this program, it will wait for input from the keyboard (since this is where standard input comes from by default)~~
−	~~$ ./avg~~
−
−	~~You can enter as many numbers as you like. Pressing Ctrl-D will break the read loop and cause the program~~ to ~~output the average value~~ of ~~the numbers you enter~~. This ~~would be a fairly tedious program to use, having to type in every number you want to average, however, you can send input to your program through a pipe.~~
−	~~$ echo -e "1\n3\n5\n7\n5\n3"~~
−	1
−	3
−	5
−	7
−	5
−	3
−
−	~~$ echo -e "1\n3\n5\n7\n5\n3" \| ./avg~~
−	4
−
−	BAM! The pipe magically sends the standard output of the <code>echo</code> command to your program. We have used <code>echo</code> here as a simple example, but you could use <code>./avg</code> to compute the average of any program that output a string of numbers, one per line.
−
−	~~==Text Manipulation Commands==~~
−	~~Linux has a rich command line history. Many (most) of the command used at the command line are just re-implementations of commands~~ that ~~were written 30 or 40 years ago. There~~ are ~~many~~ useful ~~text manipulation command available that can be combined using pipes to create "new" commands on the fly~~. ~~Here is a list of some commonly used text manipulation commands~~
−
−	~~;sort~~
−	~~:sorts text read from stdin and outputs result to stdout~~
−	~~:usefull options~~
−	~~: -r : reverse sort (sort in descending order)~~
−	~~: -k i : sort input on the i'th column (by default input is sorted on the first column)~~
−	~~: -n : interpret input as numbers and sort based on numeric value, not ascii value of text~~
−	~~: -g : interpret input as general numbers and sort based on numeric value (this will sort numbers in scientific notation)~~

@@ Line 59: / Line 59: @@
 Linux has a rich command line history. Many (most) of the command used at the command line are just re-implementations of commands that were written 30 or 40 years ago. There are many useful text manipulation command available that can be combined using pipes to create "new" commands on the fly. Here is a list of some commonly used text manipulation commands
-:sort
+;sort
-;sorts text read from stdin and outputs result to stdout
+:sorts text read from stdin and outputs result to stdout
-;usefull options
+:usefull options
-; -r    : reverse sort (sort in descending order)
+: -r    : reverse sort (sort in descending order)
-; -k i  : sort input on the i'th column (by default input is sorted on the first column)
+: -k i  : sort input on the i'th column (by default input is sorted on the first column)
-; -n    : interpret input as numbers and sort based on numeric value, not ascii value of text
+: -n    : interpret input as numbers and sort based on numeric value, not ascii value of text
-; -g    : interpret input as general numbers and sort based on numeric value (this will sort numbers in scientific notation)
+: -g    : interpret input as general numbers and sort based on numeric value (this will sort numbers in scientific notation)

← Older revision		Revision as of 17:44, 17 September 2011
Line 55:		Line 55:

	BAM! The pipe magically sends the standard output of the <code>echo</code> command to your program. We have used <code>echo</code> here as a simple example, but you could use <code>./avg</code> to compute the average of any program that output a string of numbers, one per line.		BAM! The pipe magically sends the standard output of the <code>echo</code> command to your program. We have used <code>echo</code> here as a simple example, but you could use <code>./avg</code> to compute the average of any program that output a string of numbers, one per line.
		+
		+	==Text Manipulation Commands==
		+	Linux has a rich command line history. Many (most) of the command used at the command line are just re-implementations of commands that were written 30 or 40 years ago. There are many useful text manipulation command available that can be combined using pipes to create "new" commands on the fly. Here is a list of some commonly used text manipulation commands
		+
		+	:sort
		+	;sorts text read from stdin and outputs result to stdout
		+	;usefull options
		+	; -r : reverse sort (sort in descending order)
		+	; -k i : sort input on the i'th column (by default input is sorted on the first column)
		+	; -n : interpret input as numbers and sort based on numeric value, not ascii value of text
		+	; -g : interpret input as general numbers and sort based on numeric value (this will sort numbers in scientific notation)

@@ Line 20: / Line 20: @@
 Pipes simply connect the standard output of one command to the standard input of another command. In fact, pipes are completely separate from the commands, i.e. you can use pipes with programs you write. Since pipes work with stdin and stdout, you can write a program that reads from stdin and writes to stdout and use your programs with pipes. For example, consider a simple C++ program that reads numbers from standard input and computes their average,
   #include <iostream>
@@ Line 37: / Line 38: @@
     std::cout<<sum / N;
+  }

← Older revision	Revision as of 15:25, 19 February 2013
(No difference)

@@ Line 17: / Line 17: @@
 2
 3
-The -k 2 option tells sort to sort the data based on column 2. -g tells it to sort based on numerical value
+The -k 2 option tells sort to sort the data based on column 2. -g tells it to sort based on numerical value, rather than text (ascii) value.

@@ Line 4: / Line 4: @@
 One of the most powerful features of Unix, and therefore Linux, is pipes. Pipes direct the standard output of one program to the standard input of another program. So, for exmaple, the <code>cat</code> command prints the contents of a file to standard output
   $ cat file.txt
-2
+4
 3
-4
+2
 In this example, the file named file.txt contains three lines. The <code>cat</code> command prints these lines to standard output. Now, the <code>grep</code> command reads lines from standard input, and prints lines that match a pattern to standard output. So, if we wanted to see only the lines in file.txt that contain the number 3, we can ''pipe'' the standard output of the <code>cat</code> command to <code>grep</code>
   $ cat file.txt | grep "3"
 3
-4
+2

← Older revision		Revision as of 21:42, 6 September 2011
Line 8:		Line 8:
	3 4		3 4

−	In this example, the file named file.txt contains three lines. The <code>cat</code> command prints these lines to standard output. Now, the <code>grep</code> command reads lines from standard input, and prints lines that match a pattern to standard output.	+	In this example, the file named file.txt contains three lines. The <code>cat</code> command prints these lines to standard output. Now, the <code>grep</code> command reads lines from standard input, and prints lines that match a pattern to standard output. So, if we wanted to see only the lines in file.txt that contain the number 3, we can ''pipe'' the standard output of the <code>cat</code> command to <code>grep</code>
		+	$ cat file.txt \| grep "3"
		+	2 3
		+	3 4

← Older revision		Revision as of 15:07, 6 September 2011
Line 1:		Line 1:
	Scripting refers to the practice of writing "scripts" to perform tasks on a machine. A shell script is a file that contains a set of command that well be executed as if they were typed in at the command line. Shell scripts make use of the wide range of command line tools that already exist for processing text and data. Scripts have been used by system administrators for years to automate repetitive task, but scripting has many applications in computational physics and high performance computing.		Scripting refers to the practice of writing "scripts" to perform tasks on a machine. A shell script is a file that contains a set of command that well be executed as if they were typed in at the command line. Shell scripts make use of the wide range of command line tools that already exist for processing text and data. Scripts have been used by system administrators for years to automate repetitive task, but scripting has many applications in computational physics and high performance computing.
		+
		+	==Pipes (the case for stdin)==
		+	One of the most powerful features of Unix, and therefore Linux, is pipes. Pipes direct the standard output of one program to the standard input of another program. So, for exmaple, the <code>cat</code> command prints the contents of a file to standard output
		+	$ cat file.txt
		+	1 2
		+	2 3
		+	3 4
		+
		+	In this example, the file named file.txt contains three lines. The <code>cat</code> command prints these lines to standard output. Now, the <code>grep</code> command reads lines from standard input, and prints lines that match a pattern to standard output.

BASH Scripting - Revision history

Cclark at 15:31, 19 February 2013

Cclark: moved Scripting to BASH Scripting: page will cover only bash shell scripting

Cclark: Replaced content with "bash supports many features available in traditional programming languages. However, since bash is a shell first, and programming language second, t…"

Cclark: /* Text Manipulation Commands */

Cclark at 17:44, 17 September 2011

Cclark: /* Pipes (the case for stdin) */

Cclark: /* Pipes (the case for stdin) */

Cclark: /* Pipes (the case for stdin) */

Cclark: /* Pipes (the case for stdin) */

Cclark at 15:07, 6 September 2011

Cclark: Replaced content with "`bash` supports many features available in traditional programming languages. However, since `bash` is a shell first, and programming language second, t…"