Difference between revisions of "Linux"

From PHYSpedia
Jump to: navigation, search
 
(62 intermediate revisions by 2 users not shown)
Line 1: Line 1:
[[Scripting]]
+
==Misc Links==
 +
These topics have their own pages.
 +
 
 
[[Gnuplot]]
 
[[Gnuplot]]
 +
 +
[[Common C, C++, FORTRAN Libraries]]
 +
 +
[http://en.wikibooks.org/wiki/Octave_Programming_Tutorial Octave Tutorial]
 +
 +
[[LaTeX|Latex]]
 +
 +
[[Power User Tools]]
 +
 +
==The Command Line==
 +
===Useful Commands===
 +
The following is a (short) list of useful command when manipulating data in text files. Some common options are listed, but may not be explained. For full descriptions of each command, see the <code>man</code> page. stdin, stdout, and stderr refer to standard input, output and error, respectively.
 +
 +
====File Management: Creating, Viewing, Moving, Deleting, Etc.====
 +
;<code>ls [dir]</code>
 +
: list the contents of a directory to stdout
 +
: if no directory is given, the contents of the current directory are listed
 +
: common options: -l, -a
 +
 +
;<code>cd [dir]</code>
 +
: change current working directory to a directory. i.e. go into a directory
 +
: used to navigate the filesystem
 +
 +
;<code>mkdir dir</code>
 +
: make directory named 'dir'
 +
 +
;<code>rm file1 [file2 [...] ]</code>
 +
: remove one or more files
 +
: if trying to remove a directory, the -r option must be given
 +
 +
;<code>mv source [source2 [...] ] dest</code>
 +
: move a file or files. if multiple files are given, then the last argument must be a directory.
 +
: the move command will do different things depending on how many arguments are given, and whether the arguments are files or directories.
 +
: see a list of examples [[Command_Line_Examples:_mv|here]]
 +
 +
;<code>cp source [source2 [...] ] dest</code>
 +
: copy a file or files. if multiple files are given, then the last argument must be a directory.
 +
: like the move command, <code>cp</code> will do different things depending on the arguments given. see the examples for <code>mv</code> to see how arguments determine the outcome.
 +
 +
;<code>cat [file1 [ file2 [...] ] ]</code>
 +
: print text file(s) to stdout. if multiple files are given, their text is concatenated
 +
: useful for starting a pipeline of commands from the contents of a text file
 +
 +
;<code>find dir [options]</code>
 +
: find files and directories
 +
: very powerful search utility for listing all files/directories below a given directory
 +
: useful options -name,-type
 +
 +
====Processing Text Files====
 +
;<code>cat [file1 [ file2 [...] ] ]</code>
 +
: print text file(s) to stdout. if multiple files are given, their text is concatenated
 +
: useful for starting a pipeline of commands from the contents of a text file
 +
 +
;<code>grep "pattern" [ file1 [ file2 [...] ] ]</code>
 +
: find text matching 'pattern' in files, or from stdin, and print matching lines to stdout
 +
: useful for finding a file containing a specific configuration options
 +
: useful for filtering output of a program to only those lines that are interesting
 +
: see a list of examples [[Command Line Examples: grep|here]]
 +
 +
;<code>echo [args...]</code>
 +
: print arguments to stdout
 +
: useful for starting a pipeline of commands from a variable's value
 +
: useful for printing output to the terminal from inside a script
 +
 +
;<code>sort</code>
 +
: read from stdin, sort lines, and write to stdout
 +
: common options: -k, -t, -n, -g
 +
 +
;<code>cut</code>
 +
: read from stdin and write selected columns to stdout.
 +
: common options: -d, -f
 +
 +
;<code>column</code>
 +
: read from stdin and write to stdout such that columns are aligned
 +
: common options: -t
 +
 +
;<code>head [file]</code>
 +
: read from stdin, or from a file, and print the first 10 lines
 +
: useful for taking a peak at the contents of a large file
 +
: useful for limitng the output of a pipeline
 +
: common options: -n
 +
 +
;<code>tail [file]</code>
 +
: read from stdin, or from a file, and print the last 10 lines
 +
: similar uses to head
 +
: can be used with head to print a single line, or range of lines, in a file
 +
: common options: -n
 +
 +
;<code>sed 's/old/new/g' [file]</code>
 +
: read from stdin, or from a file, and replace all instances of 'old' with 'new', then print to stdout
 +
 +
;<code>gawk '{print $1,$2}' [file]</code>
 +
: read from stdin, or from a file, and print columns (in this example, we would be printing columns 1 and 2).
 +
: <code>gawk</code> is often more useful than <code>cut</code> because it considers columns/fields to be separated by any white space
 +
: <code>gawk</code> can be used to re-order columns easily
 +
: <code>gawk</code> can be used to "transform" columns easily (for example, multiply a column by 2)
 +
: <code>gawk</code> gawk can be used to do ''much'' more than just print columns from a file. see a list of examples [[Command Line Examples: gawk |here]]
 +
 +
===Pipes===
 +
Pipes allow you to use multiple commands together. Any command that reads input from stdin and writes to stdout can be used in a pipe.
 +
So, for example, if we wanted to sort a file and print it to the screen, we could do this two ways.
 +
 +
Either just use the sort command
 +
<code>
 +
user@host:~$ sort file.txt
 +
</code>
 +
or use <code>cat</code> and <code>sort</code> together with a pipe.
 +
<code>
 +
user@host:~$ cat file.txt | sort
 +
</code>
 +
 +
We will most often use <code>cat</code> to start a pipeline of commands.
 +
 +
If we only wanted the first 10 lines of the sorted output, all we have to do is send it to <code>head</code>
 +
<code>
 +
user@host:~$ cat file.txt | sort | head
 +
</code>
 +
 +
If we only wanted the first 5 lines, we just tell <code>head</code> to show only 5 lines.
 +
<code>
 +
user@host:~$ cat file.txt | sort | head -n 5
 +
</code>
 +
 +
If we wanted to see the ''last'' 5 lines of sorted output, we could ''either'' reverse sort and use head, ''or'' use tail
 +
  <code>
 +
  user@host:~$ cat file.txt | sort -r | head -n 5
 +
  user@host:~$ cat file.txt | sort    | tail -n 5
 +
  </code>
 +
 +
===Shell Scripts===
 +
Shell scripts are just text files that contain a list of commands for the shell to run. Anything you can run at the command line can be ran in a shell script.
 +
To create a shell script, you must do two things.
 +
# create a text file with the comment <code>#! /bin/bash</code> on the first line.
 +
# make the text file "executable" with the <code>chmod</code> command.
 +
Here is an example of a simple "hello world" script.
 +
<code>
 +
#! /bin/bash
 +
 +
echo "hello world"
 +
</code>
 +
If you save this in a file named 'hello.sh', then to run it you must first make it executable.
 +
<code>
 +
user@host:~$ chmod +x hello.sh
 +
user@host:~$ ./hello.sh
 +
</code>
 +
 +
See the page on <code>bash</code> shell scripting [[BASH Scripting|here]] for information on writing scripts.

Latest revision as of 19:41, 3 May 2017

Misc Links

These topics have their own pages.

Gnuplot

Common C, C++, FORTRAN Libraries

Octave Tutorial

Latex

Power User Tools

The Command Line

Useful Commands

The following is a (short) list of useful command when manipulating data in text files. Some common options are listed, but may not be explained. For full descriptions of each command, see the man page. stdin, stdout, and stderr refer to standard input, output and error, respectively.

File Management: Creating, Viewing, Moving, Deleting, Etc.

ls [dir]
list the contents of a directory to stdout
if no directory is given, the contents of the current directory are listed
common options: -l, -a
cd [dir]
change current working directory to a directory. i.e. go into a directory
used to navigate the filesystem
mkdir dir
make directory named 'dir'
rm file1 [file2 [...] ]
remove one or more files
if trying to remove a directory, the -r option must be given
mv source [source2 [...] ] dest
move a file or files. if multiple files are given, then the last argument must be a directory.
the move command will do different things depending on how many arguments are given, and whether the arguments are files or directories.
see a list of examples here
cp source [source2 [...] ] dest
copy a file or files. if multiple files are given, then the last argument must be a directory.
like the move command, cp will do different things depending on the arguments given. see the examples for mv to see how arguments determine the outcome.
cat [file1 [ file2 [...] ] ]
print text file(s) to stdout. if multiple files are given, their text is concatenated
useful for starting a pipeline of commands from the contents of a text file
find dir [options]
find files and directories
very powerful search utility for listing all files/directories below a given directory
useful options -name,-type

Processing Text Files

cat [file1 [ file2 [...] ] ]
print text file(s) to stdout. if multiple files are given, their text is concatenated
useful for starting a pipeline of commands from the contents of a text file
grep "pattern" [ file1 [ file2 [...] ] ]
find text matching 'pattern' in files, or from stdin, and print matching lines to stdout
useful for finding a file containing a specific configuration options
useful for filtering output of a program to only those lines that are interesting
see a list of examples here
echo [args...]
print arguments to stdout
useful for starting a pipeline of commands from a variable's value
useful for printing output to the terminal from inside a script
sort
read from stdin, sort lines, and write to stdout
common options: -k, -t, -n, -g
cut
read from stdin and write selected columns to stdout.
common options: -d, -f
column
read from stdin and write to stdout such that columns are aligned
common options: -t
head [file]
read from stdin, or from a file, and print the first 10 lines
useful for taking a peak at the contents of a large file
useful for limitng the output of a pipeline
common options: -n
tail [file]
read from stdin, or from a file, and print the last 10 lines
similar uses to head
can be used with head to print a single line, or range of lines, in a file
common options: -n
sed 's/old/new/g' [file]
read from stdin, or from a file, and replace all instances of 'old' with 'new', then print to stdout
gawk '{print $1,$2}' [file]
read from stdin, or from a file, and print columns (in this example, we would be printing columns 1 and 2).
gawk is often more useful than cut because it considers columns/fields to be separated by any white space
gawk can be used to re-order columns easily
gawk can be used to "transform" columns easily (for example, multiply a column by 2)
gawk gawk can be used to do much more than just print columns from a file. see a list of examples here

Pipes

Pipes allow you to use multiple commands together. Any command that reads input from stdin and writes to stdout can be used in a pipe. So, for example, if we wanted to sort a file and print it to the screen, we could do this two ways.

Either just use the sort command


user@host:~$ sort file.txt

or use cat and sort together with a pipe.


user@host:~$ cat file.txt | sort

We will most often use cat to start a pipeline of commands.

If we only wanted the first 10 lines of the sorted output, all we have to do is send it to head


user@host:~$ cat file.txt | sort | head

If we only wanted the first 5 lines, we just tell head to show only 5 lines.


user@host:~$ cat file.txt | sort | head -n 5

If we wanted to see the last 5 lines of sorted output, we could either reverse sort and use head, or use tail

 
 user@host:~$ cat file.txt | sort -r | head -n 5
 user@host:~$ cat file.txt | sort    | tail -n 5
 

Shell Scripts

Shell scripts are just text files that contain a list of commands for the shell to run. Anything you can run at the command line can be ran in a shell script. To create a shell script, you must do two things.

  1. create a text file with the comment #! /bin/bash on the first line.
  2. make the text file "executable" with the chmod command.

Here is an example of a simple "hello world" script.


#! /bin/bash

echo "hello world"

If you save this in a file named 'hello.sh', then to run it you must first make it executable.


user@host:~$ chmod +x hello.sh
user@host:~$ ./hello.sh

See the page on bash shell scripting here for information on writing scripts.