Shell Utilities

Contents

Shell Utilities#

Table of Contents#

Utilities#

  • [ h ][ w ] ltrace

  • [ h ][ w ] sqlmap

arch - print machine hardware name (same as uname -m)

base64 - base64 encode/decode data and print to standard output

getent - get entries from Name Service Switch libraries

host - DNS lookup utility

last, lastb - show a listing of last logged in users

netstat - Print network connections, routing tables, interface statistics, masquerade connections, and multicast memberships

nl - number lines of files

od - dump files in octal and other formats

sum - checksum and count the blocks in a file

ab#

[ d ] ab - Apache HTTP server benchmarking tool

ack#

[ h ] ack

ag#

[ g ] ag

alias#

[ w ] alias

type alias

Output: alias is a shell builtin

ar#

[ w ] ar “archiver”

arch#

arch “architecture” - print architecture type or run selected architecture of a universal binary

asciiquarium#

[ h ] asciiquarium

at#

[ w ] at

awk#

[ w ] awk - pattern-directed scanning and processing language

awk is a programming language designed for processing text. an awk program consists of patterns and the corresponding code to be executed when that pattern is matched.

Options

  • -f read in the awk program from a specified file

  • -F specify a custom field delimiter (space, by default)

[example]

# awkusers.txt
Mike Jones
John Smith
Kathy Jones
Jane Kennedy
Tim Scott

awk iterates over each line of the input file and reads in each word separated by whitespace by default into a field. Field $0 represents the line itself, $1 the first word, $2 the second word, etc.

awk '$2 == "Jones" {print $0}' awkusers.txt # print each line where the user's last name is Jones
awk ' /Jones/ {print $0}' awkusers.txt      # print each line where the regex matches anywhere in the input line, not just the second word

[example]

Check each user’s password hash in csvex.txt against the dictionary file of known passwords passwords.txt.

# csvex.txt
"name","username","phone","password hash"
"John Smith","jsmith","555-555-1212",5f4dcc3b5aa765d61d8327deb882cf99
"Jane Smith","jnsmith","555-555-1234",e10adc3949ba59abbe56e057f20f883e
"Bill Jones","bjones","555-555-6789",d8578edf8458ce06fbc5bb76a58c5ca4

# passwords.txt
password,md5hash
123456,e10adc3949ba59abbe56e057f20f883e
password,5f4dcc3b5aa765d61d8327deb882cf99
welcome,40be4e59b9a2a2b5dffb918c0e86b3d7
ninja,3899dcbab79f92af727c2190bbd8abc5
abc123,e99a18c428cb38d5f260853678922e03
123456789,25f9e794323b453885f5181f1b624d0b
12345678,25d55ad283aa400af464c76d713c07ad
sunshine,0571749e2ac330a7455809c6b0e7af90
princess,8afa847f50a716e64932d995c8e7435a
qwerty,d8578edf8458ce06fbc5bb76a58c5ca4

awk -F "," '{print $4}' csvex.txt
grep "$(!!)" passwords.txt

123456,e10adc3949ba59abbe56e057f20f883e
password,5f4dcc3b5aa765d61d8327deb882cf99
qwerty,d8578edf8458ce06fbc5bb76a58c5ca4

base32#

base32 - base32 encode/decode data and print to standard output

base64#

base64 is used to encode data using the Base64 format: https://datatracker.ietf.org/doc/html/rfc4648.

options

  • -d decode Base64-encoded data

type -p base64 # /usr/bin/base64
type -t base64 # file

[example]

echo 'Rapid Cybersecurity Ops' | base64

UmFwaWQgQ3liZXJzZWN1cml0eSBPcHMK

echo 'UmFwaWQgQ3liZXJzZWN1cml0eSBPcHMK' | base64 -d

Rapid Cybersecurity Ops

basename#

[ w ] basename

basenc#

basenc - encode/decode data and print to standard output

bat#

[ g ] bat - a cat(1) clone with syntax highlighting and Git integration

batch#

[ w ] batch

bc#

[ d ][ w ] bc - arbitrary-precision decimal arithmetic language and calculator (Nelson 2006)

bg#

[ w ] bg “background”

  • man bg How do I use command bg?

bindkey#

bindkey

  • bindkey -e emacs key bindings

  • bindkey -v vi key bindings

type bindkey

Output: bindkey is a shell builtin

boxes#

[ h ][ g ] boxes

btop#

[ g ] btop

bzip2#

[ w ] bzip2

cal#

[ w ] cal, ncal - displays a calendar

cat#

[ w ] cat - concatenate and print files (Granlund & Stallman 2012)

  • man cat How do I use command cat?

  • cat /usr/share/misc/birthtoken

  • cat /usr/share/misc/flowers

cd#

[ w ] cd “change the working directory”

  • man cd How do I use command cd?

  • cd Move to the home directory.

  • cd ~ Move to the home directory.

  • cd - Move to the previous directory.

  • cd .. Move to the parent directory of the working directory.

  • cd . Stay right where we are!

cewl#

[ h ][ g ] cewl Custom Word List Generator

options

  • -a handle form-based authentication

  • -d, --depth n set the crawl depth to spider n links deep

  • -m, --min_word_length set the minimum word length

  • -o, --offsite spider external sites

  • -w, --write write the output to a file

  • -x, --max_word_length set the maximum word length

  • --extension append custom extensions to each word

  • --with-numbers accept words with numers

cheat#

  • [ g ] cheat

cheat.sh#

[ g ] https://cheat.sh/ or https://cht.sh/

  • curl https://cheat.sh/curl

chgrp#

[ w ] chgrp

chmod#

[ w ] chmod - modify file permissions

  • -f suppress error messages

  • -R recurse

[examples]

chmod u+x file
chmod 644 file

chown#

[ w ] chown - modify file owner and file group

  • -f suppress error messages

  • -R recurse

chpass#

chpass “change password” - add or change user database information

chrootkit#

[ h ][ w ] chrootkit

chsh#

chsh “change shell”

  • chsh -s /bin/zsh

  • chsh -s /bin/bash

cksum#

[ w ] cksum, sum - display file checksums and block counts

cmatrix#

[ g ] cmatrix

cmp#

[ w ] cmp “compare (byte-by-byte)” - compare two files byte by byte

  • man cmp How do I use command cmp?

column#

comm#

[ w ] comm “common” - select or reject lines common to two files

  • man comm How do I use command comm?

command#

[ w ] command

compress#

[ w ] compress

cowsay#

[ w ] cowsay

cp#

[ w ] cp “copy” - copy files (Granlund, MacKenzie, & Meyering 2012)

  • man cp How do I use command cp?

crontab#

[ w ] crontab “cronos table” - maintain crontab files for individual users (V3)

Located at /var/spool/cron.

options

  • -e edit the cron table

  • -l list the current cron table

  • -r remove the current cron table

examples

  • crontab -l List the current user’s table.

  • crontab -e Edit the current user’s table.

csplit#

[ w ] csplit

csvkit#

[ d ][ g ] csvkit

  • csvgrep

  • csvlook

  • in2csv

  • sql2csv

ctop#

  • [ y ] 01-20-2024. Techdox. “Docker Monitoring Made Easy: Quick Start with ctop”.

curl#

[ d ][ w ] curl - transfer a URL

curl is used to transfer data over a network between a client and a server and supports protocols like HTTP, HTTPS, FTP, SFTP, and TELNET.

options

  • -A specify the HTTP user agent string to send to the server

  • -d data to send with an HTTP POST request

  • -G use an HTTP GET request to send data rather than a POST

  • -I fetch only the protocol (HTTP, FTP) header

  • -L follow redirects

  • -s do not show error messages or progress bar

[example]

curl https://www.digadel.com                           # display the contents of the web page to STDOUT by default
curl https://www.digadel.com > outfile                 # put the contents of the web page in `outfile`
curl -ILs http://bitly.com/1k5eYPw | grep '^Location:' # expand a potentially-dangerous shortened URL

options

  • -I gather header information from the server (identify the web server version or operating system)

  • -L follow HTTP redirects if the page’s location has changed

  • -o send output to a file instead of STDOUT

  • -s silent

advanced usage

  • remote authentication

  • session cookies

limitations

  • single web page

[examples]

curl -Lo output.html https://www.oreilly.com
curl -LI https://www.oreilly.com
curl -LIs https://www.oreilly.com | grep '200 OK' # check whether a web site is available

Resources#

[ h ][ w ] curl

[ h ] Stenberg, Daniel. Everything curl.

  • [ y ] 01-23-2024. Navek. “curl: A Practical Guide”.

cut#

[ w ] cut - cut out selected portions of each line of a file

cut is used to extract select portions of a file. It reads a supplied input file line by line and parses the line based on a specified character. If no delimiter is specified, the tab character is used by default. The delimiter characters divide each line of a file into fields. The field number or the character position number can be used to extract parts of the file, and start at position 1.

options

  • -c specify the character(s) to extract

  • -d specify the character used as a field delimiter (default, tab)

  • -f specify the field(s) to extract

[example]

Extract users’ login names and shells from the system passwd(5) file as “name:shell” pairs.

cut -d : -f1,7 /etc/passwd

[example]

Show the names and login times of the currently logged in users.

who | cut -c 1-16,26-38

[example]

# cutfile.txt
12/05/2017 192.168.10.14 test.html
12/30/2017 192.168.10.185 login.html

cut -d' ' -f2 cutfile.txt

192.168.10.14
192.168.10.185

[example]

# cutfile.txt - columns separated by a different number of spaces
Pat····25␊
Pete···12␊

cut -d' ' -f4 cutfile.txt


12

cut -d' ' -f5 cutfile.txt

25


# cutfile.txt - columns separated by a single tab
Pat↹25␊
Pete├──┤12␊

cut -f2 cutfile.txt

25
12

[example]

# csvex.txt
"name","username","phone","password hash"
"John Smith","jsmith","555-555-1212",5f4dcc3b5aa765d61d8327deb882cf99
"Jane Smith","jnsmith","555-555-1234",e10adc3949ba59abbe56e057f20f883e
"Bill Jones","bjones","555-555-6789",d8578edf8458ce06fbc5bb76a58c5ca4

cut -d',' -f1 csvex.txt

"name"
"John Smith"
"Jane Smith"
"Bill Jones"

cut -d',' -f1 csvex.txt | tr -d '"'

name
John Smith
Jane Smith
Bill Jones

cut -d',' -f1 csvex.txt | tr -d '"' | tail -n +2

John Smith
Jane Smith
Bill Jones

cut -d',' -f3 csvex.txt | cut -c2-13 | tail -n +2

555-555-1212
555-555-1234
555-555-6789

date#

[ w ] date - display or set date and time

dd#

[ w ] dd - convert and copy a file

  • dd if=/dev/ada0 of=/dev/null bs=1m Check that a disk drive contains no bad blocks.

  • dd if=/dev/ada0 of=/dev/ada0 bs=1m Do a refresh of a disk drive, in order to prevent presently recoverable read errors from progressing into unrecoverable read errors.

  • dd if=file conv=parnone of=file.txt Remove parity bit from a file.

  • dd if=file conv=pareven | cmp -x - file Check for (even) parity errors on a file.

  • dd if=/dev/cd0 of=filename.iso bs=2048 To create an image of a Mode-1 CD-ROM, which is a commonly used format for data CD-ROM disks, use a block size of 2048 bytes.

  • dd if=memstick.img of=/dev/da0 bs=1m conv=noerror,sync Write a filesystem image to a memory stick, padding the end with zeros, if necessary, to a 1MiB boundary.

delta#

[ h ][ g ] delta

  • brew install git-delta

df#

[ w ] df - display free disk space

  • man df How do I use command df?

  • df -h base-2 human-readable format

  • df -H base-10 human-readable format

  • [ y ] Learn Linux TV. 06-23-2022. “Linux Crash Course - The df and du Commands.

dialog#

[ h ][ w ] dialog

diff#

[ w ] diff “difference” - compare files line by line

  • [ y ] Learn Linux TV. 10-18-2022. “Linux Crash Course - The diff Command”.

dig#

dirname#

[ w ] dirname

diff3#

[ w ] diff3 - compare three files line by line

dig#

[ w ] dig - DNS lookup utility

  • man dig How do I use command dig?

dmesg#

[ w ] dmesg “diagnostic messages” - display the system message buffer

  • man dmesg How do I use command dmesg?

dnstop#

[ g ] dnstop - displays various tables of DNS traffic on your network

dsniff#

[ w ] dsniff (Song, Dug)

du#

[ w ] du “disk utility” - display disk usage statistics

  • du -chs .* *

  • [ y ] Learn Linux TV. 06-23-2022. “Linux Crash Course - The df and du Commands”.

echo#

[ w ] echo - write arguments to the standard output

  • echo -n Do not print the trailing newline character.

  • echo $PATH | tr : '\n'

  • [ y ] Linode. 06-22-2022. “The Echo Command | Linux Essential Commands”.

ed#

[ w ] ed, red - line-oriented text dditor

  • man ed How do I use command ed?

entr#

[ h ][ g ] entr - run arbitrary commands when files change

  • find src/ | entr -s 'make | sed 20q' Rebuild a project if source files change, limiting output to the first 20 lines.

  • ls *.js | entr -r node app.js Launch and auto-reload a node.js server.

  • echo my.sql | entr -cp psql -f /_ Clear the screen and run a query after the SQL script is updated.

  • while sleep 0.1; do ls src/*.rb | entr -d make; done Rebuild project if a source file is modified or added to the src/ directory.

  • ls * | entr -rz ./httpd Auto-reload a web server, or terminate if the server exits.

env#

[ w ] env

eval#

eval is used to execute arbitrary command statements; eval executes the arguments in the context of the current shell.

type -p eval # ''
type -t eval # builtin

[example]

commandOne='echo'
commandArg='Hello World'
eval "$commandOne $commandArg"

Hello World

ex#

[ w ] ex

exa#

[ h ][ g ] exa - a modern replacement for ls

exit#

[ w ] exit, bye

expand#

[ w ] expand, unexpand - expand tabs to spaces, and vice versa

  • man expand How do I use command expand?

export#

export

expr#

[ w ] expr

factor#

[ w ] factor - factor numbers

false#

[ w ] false

fc#

[ w ] fc

fd#

[ g ] fd - find entries in the filesystem

  • brew install fd

ffmpeg#

[ h ] ffmpeg

mp32ogg () {
  ffmpeg -i $1 $2
}

ffuf#

[ d ][ g ] ffuf “Fuzz Faster U Fool” fast web fuzzer written in Go

Installation

brew install ffuf # macOS

fg#

[ w ] fg “foreground”

  • man fg How do I use command fg?

figlet#

[ h ] figlet

file#

[ w ] file - determine file type

file is used to help identify a given file’s type and looks deeper than the file name by reading and analyzing the first block of data known as the magic number; even if you rename a .png image file to end with .jpg the file command is smart enough to figure out and tell you the correct file type.

The file command-line utility compares the first few bytes of a file’s content to signatures in the magic pattern file in directory /usr/share/misc called magic numbers to help identify the file type.

options

  • -f read the list of files to analyze from a given file

  • -k do not stop on the first match; list all matches for the file type

  • -z look inside compressed files

“Be cautious if using the file command on an untrusted system. The file command uses the magic pattern file located at /usr/share/misc/. A malicious user could modify this file such that certain file types would not be identified. A better option is to mount the suspect drive to a known-good system and search from there.” - Cybersecurity Ops with bash

[example]

file unknownfile

unknownfile: Microsoft Word 2007+

find#

[ w ] find - walk a file hierarchy

options

  • -atime

  • -exec

  • -iname case insensitive file name

  • -maxdepth

  • -mindepth

  • -mtime

  • -name case sensitive file name

  • -print

  • -print0 print with a NUL terminator instead of a newline

  • -regex

  • -type

resources

  • [ y ] LearnLinuxTV. 04-27-2021. “Linux Essentials - The find command”.

find /c/Users -name '*password*'                          # search the Windows `/c/Users` directory and subdirectories for file names containing the word `password`
find /home -name '*password*' 2>/dev/null                 # search the Linux `/home` directory and subdirectories for file names containing the word `password` and suppress errors
find /home -name '.*'                                     # search the Linux `/home` directory and subdirectories for hidden file name
find /home -size +5G                                      # search for files greater than 5 GB in size in directory `/home` and subdirectories
find / -type f -exec ls -s '{}' \; | sort -n -r | head -5 # identify the five largest files on the system
find / -type f -exec ls -s '{}' \; | sort -n -r | tail -5 # identify the five smallest files on the system
find /home -mmin -5                                       # search for files in directory `/home` and subdirectories modified less than 5 minutes ago
find /home -mtime -1                                      # search for files in directory `/home` and subdirectories modified less than 24 hours ago
find /home -mtime +2                                      # search for files in directory `/home` and subdirectories modified more than 48 hours ago
find /home -atime -1                                      # search for files in directory `/home` and subdirectories accessed less than 24 hours ago
find /home -type f -atime -1 -exec cp '{}' ./ \;          # search for files in directory `/home` and subdirectories accessed less than 24 hours ago and copy them to the working directory (make sure the working directory is outside directory `/home`)
find . -type d -name "b*"                                 # search for directories that begin with `b`.
find . -regex '\.\/\.bash.*'                              # search for patterns that begin with `./.bash`.
find . -maxdepth 1 -iname "*abc*" ! -name "*:*" -print    #

# `-o, -or` logical or
# command `find` uses logical and `-a, -and` by default
find . -name 'a.txt' -o  -name 'b.txt' # logical or
find . -name 'a.txt' -or -name 'b.txt' # logical or

find /data -type f -exec grep '{}' -e 'ProductionWebServer' \; -exec cat '{}' >> ProductionWebServerAgg.txt \;

# users and permissions
find . -type f -user jsmith                               # find files in the working directory that are owned by user `jsmith`...
find . -type f -user jsmith -exec chown mwilson '{}' \;   # ...and change their owner to user `mwilson`
find . -type f -name '*secret*' -exec chmod 600 '{}' \;   # find files in the working directory that contain the string `secret` in their name and make them accessible only to their owner

finger#

[ w ] finger - user information lookup program

fold#

[ w ] fold - fold long lines for finite width output device

fortune#

[ w ] fortune

fsck#

[ w ] fsck - filesystem consistency check and interactive repair

ftp#

File Transfer Protocol

options

  • -n do not attempt to automatically log into the server

interactive commands

  • ls

  • cd

  • put transfer files to the FTP server

  • get transfer files from the FTP server

  • quit

[example]

ftp 192.168.0.125    # connect to an FTP server at 192.168.0.125 on port 21
ftp -n 192.168.0.125 # connect to an FTP server at 192.168.0.125 on port 21 but do not automatically login
ftp 192.168.0.125 50 # connect to an FTP server at 192.168.0.125 on port 50

fuck#

[ g ] fuck

fuser#

[ w ] fuser

fzf#

[ g ] fzf - a command-line fuzzy finder

getcat#

gencat - NLS catalog compiler

getconf#

getconf – retrieve standard configuration variables

getopts#

[ w ] getopts

git#

git (Torvalds and Hamano 2014)

glances#

[ h ] glances - an eye on your system

grep#

[ w ] grep, egrep, fgrep, rgrep, bzgrep, bzegrep, bzfgrep, zgrep, zegrep, zfgrep - file pattern searcher (Meyering 2012a)

The grep command searches the content of a file for a given pattern and prints any line where the pattern is matched.

options

  • -e regex search pattern

  • -i case insensitive

  • -n line number

  • -o only the text that matches the regex pattern will be returned instead of the entire line

  • -P Perl-specific pattern-match modifier (not available on macOS)

  • -r recurses through subdirectories

  • -w match whole words only

  • -z treats newlines like any ordinary character in its search and adds a null value (ASCII 0) at the end of each string found

[examples]

grep -i -r /home -e 'password'                                        # search for files in directory `/home` and subdirectories that contain the string `password`
find /home -type f -exec grep 'password' '{}' \; -exec cp '{}' . \; & # search for files in directory `/home` and subdirectories that contain the string `password` and copy those files to the working directory in the background
find / -type f -exec file '{}' \; | egrep 'PNG' | cut -d' ' -f1       # search the system for png files

[example]

# book.xml
<book title="Cybersecurity Ops with bash" edition="1">
  <author>
    <firstName>Paul</firstName>
    <lastName>Troncone</lastName>
  </author>
  <author>
    <firstName>Carl</firstName>
    <lastName>Albing</lastName>
  </author>
</book>

grep -o '<firstName>.*<\/firstName>' book.xml

<firstName>Paul</firstName>
<firstName>Carl</firstName>

grep -Pzo '(?s)<author>.*?</author>' book.xml # `(?s)` is a Perl-specific pattern-match modifier which modifies the `.` metacharacter to also match on the newline character

<author>
  <firstName>Paul</firstName>
  <lastName>Troncone</lastName>
</author>
<author>
  <firstName>Carl</firstName>
  <lastName>Albing</lastName>
</author>

grep -Po '<firstName>.*?<\/firstName>' book.xml | sed 's/<[^>]*>//g' # `[^>]*` means match any non-">" character zero or more times

Paul
Carl

[example]

# book.json
{
  "title": "Cybersecurity Ops with bash",
  "edition": 1,
  "authors": [
    {
      "firstName": "Paul",
      "lastName": "Troncone"
    },
    {
      "firstName": "Carl",
      "lastName": "Albing"
    }
  ]
}

grep -o '"firstName": ".*"' book.json # extract firstName key-value pairs

"firstName": "Paul"
"firstName": "Carl"

grep -o '"firstName": ".*"' book.json | cut -d" " -f2 | tr -d '\"'

Paul
Carl

groups#

groups – show group memberships

  • id -Gn equivalent to groups

gzip#

[ w ] gzip, gunzip, gzcat, zcat - compression/decompression tool using Lempel-Ziv coding (LZ77)

  • man gzip How do I use command gzip?

  • man gunzip How do I use command gunzip?

  • gzip archive.tar Compress tarball archive.tar.

hash#

[ w ] hash

hashcat#

  • [ h ][ g ][ w ] hashcat

  • [ h ][ g ][ w ] hashcat-utils

[example]

hashcat --help | grep -i macos

    122 | macOS v10.4, macOS v10.5, macOS v10.6                      | Operating System
   1722 | macOS v10.7                                                | Operating System
   7100 | macOS v10.8+ (PBKDF2-SHA512)                               | Operating System

hexdump#

[ w ] hexdump - ASCII, decimal, hexadecimal, octal dump

history#

[ w ] history

  • man history How do I use command history?

hollywood#

[ g ] hollywood

host#

[ w ] host

hostname#

[ w ] hostname - set or print name of current host system

  • man hostname How do I use command hostname?

  • hostname -s Print the host name.

  • hostname -d Print the host name’s domain.

htop#

[ h ][ g ][ w ] htop - interactive process viewer

  • man htop How do I use command htop?

hub#

[ g ] hub

hydra#

[examples]

hydra -U http-post-form

hyperfine#

[ g ] hyperfine

iconv#

[ w ] iconv

ifconfig#

[ w ] ifconfig “interface configuration” - configure network interface parameters

  • man ifconfig How do I use command ifconfig?

iostat#

[ w ] iostat - report I/O statistics

  • iostat -w 1 disk0 disk2 Display statistics for the first and third disk devices device every second ad infinitum.

  • iostat -c 2 Display the statistics for the first four devices in the system twice, with a one second display interval.

  • iostat -Iw 3 Display total statistics every three seconds ad infinitum.

  • iostat -odICTw 2 -c 9 Display total statistics using the old-style output format 9 times, with a two second interval between each measurement/display. The -d flag generally disables the TTY and CPU displays, but since the -T and -C flags are given, the TTY and CPU displays will be displayed.

iotop#

[ w ] iotop - display top disk I/O events by process. uses DTrace.

ip#

ip - Internet Protocol

  • man ip How do I use command ip?

# an address (a) object represents the IPv4/IPv6 address(es) associated with a link
ip     a                  # display                               all links
ip -br a                  # display the state and IP addresses of all links
ip     a show <interface> # display a particular link

# a link (l) object represents a physical or logical network device
ip       l                  # display all interfaces
ip -s    l                  # display all interfaces - with statistics
ip -s -s l
ip -br   l
ip       l show <interface> # display a particular interface
ip       l show up

ip -br      l show up | sort
ip -s -s -h l show br0

# an maddress (m) object represents the multicast address associated with a link
ip maddress

ip monitor

# a neighbor (n) object represents information about a neighboring interface (cached ARP or IPv6 neighbor discovery (NDISC))
ip neighbor

# a rule (ru) object represents a policy rule for routing packets and controls the precedence of the routes in the routing database
ip rule

# a route (r) object represents an entry in the routing table
ip route

 10. 0.  0.0/24 dev br0       proto kernel scope link src 10.0.0.1
 10. 0.  2.0/24 dev br2       proto kernel scope link src 10.0.2.1
 10. 0.  4.0/24 dev br4       proto kernel scope link src 10.0.4.1
 10. 0.  6.0/24 dev br6       proto kernel scope link src 10.0.6.1
 10. 0.  8.0/24 dev wgsrv1    proto kernel scope link src 10.0.8.1
 10. 0. 10.0/24 dev wgsrv2    proto kernel scope link src 10.0.10.1
100.14. 54.0/24 dev eth8      proto kernel scope link src 100.14.54.175
203. 0.113.0/24 dev dnsfilter proto kernel scope link src 203.0.113.1

ipconfig#

[ w ] ipconfig – view and control IP configuration state

ipcrm#

[ w ] ipcrm – remove the specified message queues, semaphore sets, and shared segments

ipcs#

[ w ] ipcs

iptables#

[ w ] iptables

jobs#

jobs

  • man jobs How do I use command jobs?

john#

  • [ h ][ g ] john John the Ripper

  • [ w ] RockYou

brew install john-jumbo # brew install john

# `hashlist.txt` or `/etc/shadow`
admin:e08e4506d2e3f370a5e8ab79647df309
guest:a132mj06d2e3f370a5e8ab79647df309

john hashlist.txt
john hashlist.txt -wordlist=wordlist.txt
john --show hashlist.txt

join#

[ w ] join

join combines the lines of two files that share a common field. In order for join to work properly the input files must be sorted.

Options

  • -j join using the specified field number; fields start at 1

  • -1 n -2 m join the nth column of file 1 to the mth column of file 2 (defaults to -1 1 -2 1)

  • -t specify the character to use as the field separator; space is the default field separator

  • --header use the first line of each file as a header

[example]

# usernames.txt
1,jdoe
2,puser
3,jsmith

# accesstime.txt
0745,file1.txt,1
0830,file4.txt,2
0830,file5.txt,3

join -1 3 -2 1 -t, accesstime.txt usernames.txt
join -1 3 -t, accesstime.txt usernames.txt

1,0745,file1.txt,jdoe
2,0830,file4.txt,puser
3,0830,file5.txt,jsmith

[example]

# ips.txt
ip,OS
10.0.4.2,Windows 8
10.0.4.35,Ubuntu 16
10.0.4.107,macOS
10.0.4.145,macOS

# users.txt
user,ip
jdoe,10.0.4.2
jsmith,10.0.4.35
msmith,10.0.4.107
tjones,10.0.4.145

# join the second column of the second file
join -t, -1 1 -2 2 ips.txt users.txt
join -t, -2 2 ips.txt users.txt

ip,OS,user
10.0.4.2,Windows 8,jdoe
10.0.4.35,Ubuntu 16,jsmith
10.0.4.107,macOS,msmith
10.0.4.145,macOS,tjones

jot#

jot - print sequential or random data

  • man jot How do I use command jot?

  • jot - 1 10 prints the integers from 1 to 10

  • jot 21 -1 1.00 prints 21 evenly spaced numbers increasing from -1 to 1

  • jot -c 128 0 generates the ASCII character set

  • jot -w xa%c 26 a generates the strings xaa through xaz

  • jot -r -c 160 a z | rs -g 0 8 generates 20 random 8-letter strings

  • jot -b yes 0 generates infinitely many yes’s

  • jot -w %ds/old/new/ 30 2 - 5 generates thirty ed(1) substitution commands applying to lines 2, 7, 12, etc.

  • jot -w %d - 9.5 0 -.5 generates stuttering sequence 9, 9, 8, 8, 7, etc. by truncating the output precision and a suitable choice of step size

  • jot -b x 512 > block generates a file containing exactly 1024 bytes

  • expand -``jot -s, - 10 132 4`` set tabs four spaces apart starting from column 10 and ending in column 132

  • grep ``jot -s "" -b . 80`` print all lines 80 characters or longer

jq#

[ h ] jq - Command-line JSON processor (Dolan 2014)

jq is a lightweight language and JSON parser.

[example]

# book.json
{
  "title": "Cybersecurity Ops with bash",
  "edition": 1,
  "authors": [
    {
      "firstName": "Paul",
      "lastName": "Troncone"
    },
    {
      "firstName": "Carl",
      "lastName": "Albing"
    }
  ]
}

jq '.title' book.json

"Cybersecurity Ops with bash"

jq '.authors[].firstName' book.json

"Paul"
"Carl"

jq '.authors[0].firstName' book.json

"Paul"

kill#

[ w ] kill - terminate or signal a process

  • man kill How do I use command kill?

  • kill 142 157 Terminate the processes with PIDs 142 and 157.

  • kill -s HUP 507 Send the hangup signal (SIGHUP) to the process with PID 507.

  • kill -- -117 Terminate the process group with PGID 117.

killall#

[ w ] killall - kill processes by name

  • man killall How do I use command killall?

  • killall firefox Send SIGTERM to all firefox processes.

  • killall -u ${USER} firefox Send SIGTERM to firefox processes belonging to USER.

  • killall -SIGSTOP firefox Stop all firefox processes.

  • killall -SIGCONT firefox Resume firefox processes.

  • killall -s firefox Show what would be done to firefox processes, but do not actually signal them.

  • killall -m 'vim*' Send SIGTERM to all processes matching provided pattern (like vim and vimdiff).

less#

[ w ] less - opposite of more

  • man less How do I use command less?

lex#

[ w ] lex

libpcap#

[ h ][ g ][ w ] libpcap

resources

ln#

[ w ] ln

locale#

locale – display locale settings

localedef#

localedef – define locale environment

login#

login – log into the computer

logout#

logout

logname#

[ w ] logname

lolcat#

[ g ] lolcat

lp#

[ w ] lp

ls#

[ w ] ls “list the contents of the directory” (Stallman & MacKenzie 2012)

This command answers the question, “What else is here in the working directory with me?”.

options

  • -a view hidden files including . and ..

  • -A view hidden files excluding . and ..

  • -F

  • -i

  • -l

  • -R recurse through subdirectories

  • -s identify the size of a file in blocks, not bytes

ls -AilsF
ls the/relative/path/to/some/directory/x # list the contents of directory x
ls / -R -s | sort -n -r | head -5        # identify the five largest files on the system
ls / -R -s | sort -n -r | tail -5        # identify the five largest files on the system

[“bare-bones implementation”] Advanced Programming in the UNIX Environment, 3e

#include "apue.h"
#include <dirent.h>

/* myls.c
 *
 * print the name of every file in a directory
 */

int main (int argc, char *argv[]) {
  DIR           *dp;
  struct dirent *dirp;

  if (argc != 2)
    err_quit("usage: ls directory_name");

  if ((dp = opendir(argv[1])) == NULL)
    err_sys("can't open %s", argv[1]);

  while ((dirp = readdir(dp)) != NULL)
    printf("%s\n", dirp->d_name);

  closedir(dp);
  exit(0);
}

cc myls.c  # the historical C compiler, usually linked to `gcc`
# or
gcc myls.c # the GNU C compilation system

./a.out /dev

lsd#

[ g ] lsd - an ls command with a lot of pretty colors and some other stuff

  • lsd --help How do I use command lsd?

lsof#

[ w ] lsof “list open files”

resources

  • [ y ] 01-03-2024. Kevan. “lsof: A Quick Practical Guide”.

lsvfs#

lsvfs - list installed virtual file systems

lynis#

[ h ][ w ] lynis - system and security auditing tool

  • man lynis How do I use command lynis?

  • brew install lynis

  • sudo apt install lynis

  • sudo lynis show help

  • sudo lynis show groups

m4#

[ w ] m4

mail#

[ w ] mail

mailx#

[ w ] mailx

make#

[ w ] make

man#

[ w ] man - display online manual documentation pages (Eaton & Watson 2014)

  • man ascii display ASCII table

man ls
man 1 ls
man -s1 ls

mc#

[ h ][ w ] mc “midnight commander” - visual shell for Unix-like systems

md5sum#

[ w ] md5sum - compute and check MD5 message digest

mesg#

[ w ] mesg

mkdir#

[ w ] mkdir “make directory” (MacKenzie 2012)

  • man mkdir How do I use command mkdir?

  • mkdir -m 700 foobar Create a directory named foobar and set its file mode to 700.

  • mkdir -p cow/horse/monkey Create a directory named cow/horse/monkey, creating any non-existent intermediate directories as necessary.

  • mkdir -pv {a,b}/{c,d}/{e,f}/{g,h}

mkfifo#

[ w ] mkfifo

molotov#

[ h ] molotov

more#

[ w ] more

  • man more How do I use command more?

mtr#

[ h ][ g ][ w ] mtr

mv#

[ w ] mv “move” - move files (Parker, MacKenzie, & Meyering 2012)

  • man mv How do I use command mv?

  • mv -i foo bar check whether another file will be overwritten before renaming a file

  • mv -f foo bar force rename a file, potentially overwriting another file

  • mv -v foo var verbose

nano#

[ w ] nano, pico - simple text editor in the style of the Alpine Composer

nc#

options

  • -l listen for incoming connections (act as a server)

  • -n do not perform a DNS lookup

  • -p the source port to connect from or listen on

  • -v verbose

[example]

nc www.oreilly.com 80 # initialize a connection to oreilly.com on destination port 80
nc -l -v -n -p 8080   # listen for incoming connections on port 8080

ncdu#

[ h ][ w ] ncdu - NCurses Disk Usage

  • brew install ncdu

  • ncdu scan and browse the directory you’re currently in

  • ncdu -x / scan a full filesystem, your root filesystem, for example

  •    # ...some time later:
   zcat export.gz | ncdu -f-``` Since scanning a large directory may take a while, you can scan a directory and export the results for later viewing. To export from a cron job, make sure to replace "-1" with "-0" to suppress any unnecessary output.

  • ncdu -o- | tee export.file | ./ncdu -f- export a directory and browse it once scanning is done

  • ncdu -o- | gzip | tee export.gz | gunzip | ./ncdu -f- the same is possible with gzip compression, but is a bit kludgey

  • ssh -C user@system ncdu -o- / | ./ncdu -f- To scan a system remotely, but browse through the files locally. The “-C” option to ssh enables compression, which will be very useful over slow links. Remote scanning and local viewing has two major advantages when compared to running ncdu directly on the remote system: You can browse through the scanned directory on the local system without any network latency, and ncdu does not keep the entire directory structure in memory when exporting, so you won’t consume much memory on the remote system.

neofetch#

[ g ] neofetch - a fast, highly customizable system info script

netstat#

[ w ] netstat - show network status

  • man netstat How do I use command netstat?

newgrp#

newgrp – change to a new group

nfsstat#

nfsstat - display NFS statistics

nice#

[ w ] nice – execute a utility at an altered scheduling priority

nl#

[ w ] nl - line numbering filter

nm#

[ w ] nm - display name list (symbol table)

nmap#

nmap - network exploration tool and security / port scanner

nohup#

[ w ] nohup – invoke a utility immune to hangups

nslookup#

[ w ] nslookup

od#

[ w ] od - octal, decimal, hex, ASCII dump

openssl#

arguments

  • aes-256-cbc specifies the 256-bit version of AES

options

  • -base64 specifies that the output be Base64-encoded

  • -in specifies the file to encrypt

  • -out specifies the file to which to output the ciphertext

  • -pass specifies the encryption key

osquery#

[ h ][ g ] osquery

paste#

[ w ] paste - merge corresponding or subsequent lines of files (Ihnat & MacKenzie 2012)

  • ls | paste - - - List the files in the current directory in three columns.

  • paste -s -d '\t\n' myfile Combine pairs of lines from a file into single lines.

  • sed = myfile | paste - - Number the lines in a file, similar to nl(1).

  • find / -name bin -type d | paste -s -d : - Create a colon-separated list of directories named bin, suitable for use in the PATH environment variable.

patch#

[ w ] patch - apply a diff file to an original

pathchk#

pathchk – check pathnames

pax#

[ w ] pax – read and write file archives and copy directory hierarchies

pfetch#

[ g ] pfetch

ping#

[ w ] ping - send ICMP ECHO_REQUEST packets to network hosts

options

  • -c the number of ping requests to send to the remote system (Linux)

  • -n the number of ping requests to send to the remote system (Windows)

  • -w time in milliseconds to wait for a reply (Windows)

  • -W time in seconds to wait for a reply (Linux)

resources

  • [ y ] Learn Linux TV. 05-31-2022. “Linux Crash Course - The ping Command”.

[examples]

ping -n1 192.168.0.11

pr#

[ w ] pr – print files

print#

[ w ] print

  • print -l print each word on a new line

printf#

[ w ] printf - formatted output

options

  • -v VAR save to a variable VAR instead of printing

examples

  • printf "%s\n" hello Print the string “hello”.

  • printf %s\n hello Same as above, but notice that the format string is not quoted and hence we do not get the expected behavior.

  • printf "%+d\n%d\n%d\n" 1 -2 13 Print arguments forcing sign only for the first argument.

  • printf "%+d\n" 1 -2 13 Same as above, but the single format string will be applied to the three arguments.

  • printf "%.2f\n" 31.7456 Print number using only two digits after the decimal point.

ps#

[ w ] ps - process status

ps auwx

pup#

[ w ] pup - a command line tool for processing HTML

  • brew install pup

pwd#

[ w ] pwd “print the working directory” - return working directory name (Meyering 2012b)

  • man pwd How do I use command pwd?

  • pwd “Where am I in the filesystem now?”

  • pwd -L Display the logical current working directory. Same as pwd

  • pwd -P Display the physical current working directory (all symbolic links resolved).

rclone#

[ h ][ g ] rclone - rclone syncs your files to cloud storage

read#

[ w ] read

reads user input from stdin and stores it in a variable

realpath#

realpath - print the resolved path

renice#

[ w ] renice – alter priority of running processes

reset#

[ w ] reset, tput - initialize a terminal or query terminfo database

rev#

rg#

[ g ] rg - recursively search the current directory for lines matching a pattern

  • brew install ripgrep

rm#

[ w ] rm “remove” – remove directory entries (Rubin et al 2012)

  • man rm How do I use command rm?

  • rm -- -filename The rm command uses getopt(3) to parse its arguments, which allows it to accept the ‘–’ option which will cause it to stop processing flag options at that point. This will allow the removal of file names that begin with a dash (‘-’).

  • rm /home/user/-filename or rm ./-filename The same behavior can be obtained by using an absolute or relative path reference.

  • rm -rf use with caution

rmdir#

[ w ] rmdir “remove directory”

  • man rmdir How do I use command rmdir?

  • rmdir -p Each directory argument is treated as a pathname of which all components will be removed, if they are empty, starting with the last most component. (See rm(1) for fully non-discriminant recursive removal.)

  • rmdir -v verbose

  • rmdir foobar Remove the directory foobar, if it is empty.

  • rmdir -p cow/horse/monkey Remove all directories up to and including cow, stopping at the first non-empty directory (if any).

rmlint#

[ d ][ g ] rmlint - find duplicate files and other space waste efficiently

rs#

rs - reshape a data array

  • man rs How do I use command rs?

rsync#

[ h ][ w ] rsync - faster, flexible replacement for rcp

  • man rsync How do I use command rsync?

  • [ y ] Linode. 06-09-2021. “How to Use the rsync Command | Linux Essentials Tutorial”.

rsyslog#

[ w ] rsyslog

rush#

[ g ] rush

sa#

sa - print system accounting statistics

sample-stream#

[ g ] sample-stream

scp#

scp “secure copy” - OpenSSH secure file copy

  • man scp How do I use command scp?

https://www.hostinger.com/tutorials/using-scp-command-to-transfer-files/#Can_I_Pass_Password_in_SCP_Command

screen#

screen - screen manager with VT100/ANSI terminal emulation

  • man screen How do I use command screen?

sdiff#

[ w ] sdiff - compares two files side by side and outputs and differences

options

  • -a treat files as text files

  • -i ignore case

  • -s suppress lines common between the two files

  • -w maximum number of characters to output per line

[example]

sdiff -s file1.txt file2.txt # compare two files and output only lines that differ

sed#

[ w ] sed “stream editor”

sed allows you to perform edits (such as replacing characters) on a stream of data

Options

  • -i edit the specified file and overwrite it in place

[example]

# ips.txt
ip,OS
10.0.4.2,Windows 8
10.0.4.35,Ubuntu 16
10.0.4.107,macOS
10.0.4.145,macOS

sed 's/10\.0\.4\.35/10.0.4.27/g' ips.txt # replace all instances of 10.0.4.35 with 10.0.4.27

Resources

seq#

[ w ] seq “sequence” - print sequences of numbers (Drepper 2012)

  • man seq How do I use command seq?

set#

set

setenv#

setenv

sh#

[ w ] sh – POSIX-compliant command interpreter

shasum#

shasum - Print or Check SHA Checksums

shutdown#

shutdown - close down the system at a given time

  • man shutdown How do I use command shutdown?

shred#

[ w ] shred - overwrite a file to hide its contents, and optionally delete it

sleep#

[ w ] sleep – suspend execution for an interval of time

slurm#

[ g ] slurm - yet another network load monitor

smartctl#

[ h ] smartctl - Control and Monitor Utility for SMART Disks

  • brew install smartmontools

  • smartctl -a disk0

sort#

[ w ] sort – sort or merge records (lines) of text and binary files (Haertel & Eggert 2012)

sort is used to rearrange a text file into numerical and alphabetical order. By default, sort arranges lines in ascending order starting with numbers and then letters; uppercase letters are placed before the corresponding lowercase unless specified otherwise.

Options

  • -f ignore case

  • -k sort based on a subset of the data (key) in a line; fields are delimited by whitespace

  • -n numeric sort: use numerical ordering (e.g., so that 1, 2, 3 all sort before 10)

  • -o write output to a specified file

  • -r reverse sort: sort in descending order

# file1.txt
12/05/2017 192.168.10.14 test.html
12/30/2017 192.168.10.185 login.html

Sort by the file name column and ignore the IP address column.

sort -k3 file1.txt

12/30/2017 192.168.10.185 login.html
12/05/2017 192.168.10.14 test.html

Sort by the second octet in the IP address.

sort -k2.5,2.7 file1.txt

12/05/2017 192.168.10.14 test.html
12/30/2017 192.168.10.185 login.html

source#

source

split#

[ w ] split – split a file into pieces

sponge#

sponge - soak up standard input and write to a file

spyql#

[ d ][ g ] spyql

sqlmap#

  • [ h ] sqlmap

ssh#

[ w ] ssh “secure shell” - OpenSSH remote login client

  • man ssh How do I use command ssh?

ssh-add#

ssh-add - adds private key identities to the OpenSSH authentication agent

ssh-agent#

[ w ] ssh-agent - OpenSSH authentication agent

ssh-copy-id#

ssh-copy-id - use locally available keys to authorise logins on a remote machine

ssh-keygen#

[ w ] ssh-keygen - OpenSSH authentication key utility

stat#

stat, readlink - display file status

strings#

[ w ] strings - find the printable strings in a object, or other binary, file

strip#

[ w ] strip - remove symbols

su#

[ w ] su “substitute user identity”

  • su -m operator poweroff Starts a shell as user operator, and runs the command poweroff. You will be asked for operator’s password unless your real UID is 0. Note that the -m option is required since user “operator” does not have a valid shell by default. In this example, -c is passed to the shell of the user “operator”, and is not interpreted as an argument to su.

  • su -m operator 'shutdown -p now' Same as above, but the target command consists of more than a single word and hence is quoted for use with the -c option being passed to the shell. (Most shells expect the argument to -c to be a single word).

  • su -l foo Simulate a login for user foo.

  • su - foo Same as above.

  • su - Simulate a login for root.

sudo#

[ w ] sudo, sudoedit - execute a command as another user

  • sudo -v && while true; do sudo -n true; sleep 60; kill -0 "$$" || exit; done 2> /dev/null & # ask for the administrator password upfront; keep-alive: update the existing sudo timestamp until the program that contains this command is done (not tested)

tail#

tac - concatenate and print files in reverse

tail#

[ w ] tail – display the last part of a file

tail is used to output the last lines of a file. By default, tail will output the last 10 lines of a file.

Options

  • -cn tail last n characters

  • -f continuously monitor the file and output lines as they are added

  • -n output the number of lines specified

  • -r reverse line order

[examples]

tail -n1 somefile.txt
tail -n+2 somefile.txt   # output the contents of the file starting at line number 2
tail -n20 /var/log/syslog
tail -f /var/log/syslog

talk#

[ w ] talk – talk to another user

tcpdump#

[ h ][ w ] tcpdump - dump traffic on a network

tee#

[ w ] tee – duplicate standard input

telnet#

[example]

telnet 192.168.0.16 25

test#

[ w ] test, [ – condition evaluation utility

time#

[ w ] time – time command execution

tldr#

[ h ] tldr

tmux#

[ d ] tmux “terminal multiplexer” [ cheat sheet ]

resources

  • [ y ] 07-15-2020. Beginbot. “Getting Starting With Tmux Isn’t That Scary”.

  • [ y ] 08-19-2023. Learn Linux TV. “Learn tmux (Part 1) Boost your Linux Productivity with this 5-part Course!”.

  • [ y ] 08-19-2023. Learn Linux TV. “Learn tmux (Part 2) - How to use Splits and Panes for the Ultimate Linux Workflow!”.

  • [ y ] 08-19-2023. Learn Linux TV. “Learn tmux (Part 3) - A Simple and Straight-Forward Look at Managing Windows”.

  • [ y ] 08-19-2023. Learn Linux TV. “Learn tmux (Part 4) - Discover how to manage Sessions within tmux”.

  • [ y ] 08-19-2023. Learn Linux TV. “Learn tmux (Part 5) - How to Customize tmux and Make It Your Own!”.

  • [ y ] 12-30-2023. NetworkChuck. “you need to learn tmux RIGHT NOW!!”.

  • [ y ] 10-27-2019. Lawrence Systems. “Getting Started With TMUX”.

  • [ y ] 06-18-2019. The Perl and Raku Conference. “Doug Bell - “Organized Development With tmux””.

more

  • [ y ] 01-21-2024. Mischa van den Burg. “My Entire Neovim + Tmux Workflow As A DevOps Engineer On MacOS”.

  • [ y ] 01-22-2023. typecraft. “Make TMUX Look Amazing in 3 Minutes! - complete tutorial”.

top#

[ w ] top “table of processes” - display sorted information about processes

  • man top How do I use command top?

  • ps -eo pcpu,pid,user,args | sort -r | head -10

touch#

[ w ] touch – change file access and modification times

tput#

[ w ] tput, reset - initialize a terminal or query terminfo database

tput, reset - initialize a terminal or query terminfo database

tput can be used to control format in the terminal (e.g., cursor location, cursor behavior, etc.)

tput is an “extraction”: it looks up terminal formatting codes in database terminfo

parameters

  • clear clear the screen

  • cols print the number of terminal columns

  • cup <x> <y> move the cursor to position <x> and <y>

  • lines print the number of terminal lines

  • rmcup restore the previously saved terminal layout

  • setab set the terminal background color

  • setaf set the terminal foreground color

  • smcup save the current termainl layout and clear the screen

type -t tput          # file
type -t /usr/bin/tput # file

tr#

[ w ] tr – translate characters

tr is used to translate or map from one character to another and is often used to delete unwanted or extraneous characters. It only reads from stdin and writes to stdout so it is typically seen with redirects for the input and output files.

Options

  • -d delete the specified characters from the input stream

  • -s “squeeze”: replace the repeated instances of a character with a single instance

[example]

In the following example, the characters from the first argument are mapped to the corresponding characters in the second argument. Two backslashes are needed to specify a single backslash character because the backslash has a special meaning to tr: it is used to indicate special characters such as newline \n, return \r, or tab \t. Single quotes are used around the arguments to avoid any special interpretation by bash.

# infile.txt
drive:path\name
c:\Users\Default\file.txt

tr '\\:' '/|' < infile.txt > outfile.txt # translate all the backslashes into forward slashes and all the colons into vertical bars

# outfile.txt
drive|path/name
c|/Users/Default/file.txt

traceroute#

[ w ] traceroute

resources

  • http://traceroute.org/

  • [ y ] 07-12-2022. CertBros. “Traceroute Explained | Real World Examples”.

  • [ y ] 01-29-2021. Computerphile. “How Traceroute Works (Building a Movie Scene ‘Trace’ Map) - Computerphile”.

  • [ y ] 03-04-2022. David Bombal. “Traceroute explained // Featuring Elon Musk // Demo with Windows, Linux, macOS”.

  • [ y ] 11-18-2020. Epic Network Labs. “What’s wrong with Traceroute? How to fix it with Paris Traceroute! | Network Traffic Analysis Ep 12”.

trap#

[ w ] trap

Options

  • -l list all supported signals

  • -p <signal> list the trap commands associated with that signal

tree#

[ w ] tree - list contents of directories in a tree-like format

  • man tree How do I use command tree?

  • tree -a -L 1 include hidden files; top level only

trim#

tripwire#

[ g ][ w ] tripwire

true#

[ w ] true – return true value

truncate#

truncate - shrink or extend the size of a file to the specified size

tsort#

[ w ] tsort – topological sort of a directed graph

tty#

[ w ] tty – return user’s terminal name

type#

[ w ] type

  • type -a <COMMANDNAME>

ulimit#

ulimit

umask#

[ w ] umask

https://www.liquidweb.com/kb/what-is-umask-and-how-to-use-it-effectively/

unalias#

unalias

uname#

[ w ] uname – display information about the system

uncompress#

[ w ] uncompress - expand data

unexpand#

[ w ] unexpand - expand spaces to tabs

unimatrix#

[ g ] unimatrix

uniq#

[ w ] uniq – report or filter out repeated lines in a file

uniq filters out duplicate lines of data that occur adjacent to one another. To remove duplicate lines in a file, be sure to sort it before using uniq.

Options

  • -c printthe number of times a line is repeated

  • -f ignore the specified number of fields before comparing; fields are delimited using spaces

  • -i ignore the case; by default, uniq is case-sensitive

unzip#

unzip - list, test and extract compressed files in a ZIP archive

  • man unzip How do I use command unzip?

uptime#

uptime – show how long system has been running

uucp#

[ w ] uucp - Unix to Unix copy

uudecode#

[ w ] uudecode - decode a binary file

uuencode#

[ w ] uuencode - encode a binary file

uustat#

uustat - UUCP status inquiry and control

vim#

[ w ] vi, vim, ex, view, gvim, gview, evim, eview, rvim, rview, rgvim, rgview - Vi IMproved, a programmer’s text editor

vimdiff#

vimdiff - edit between two and eight versions of a file with Vim and show differences

visudo#

visudo - edit the sudoers file

vnstat#

[ h ] vnstat - a console-based network traffic monitor

wait#

[ w ] wait

watch#

[ w ] watch - execute a program periodically, showing output fullscreen

wc#

[ w ] wc “word count” – word, line, character, and byte count (Rubin & MacKenzie 2012)

  • man wc How do I use command wc?

  • wc -mlw report1 report2 Count the number of characters, words and lines in each of the files report1 and report2 as well as the totals for both.

wget#

[ w ] wget - the non-interactive network downloader

  • man wget How do I use command wget?

options

  • -p download diles associated with the web site

  • -m enable mirror mode

  • -k convert links in the downloaded pages to local paths

  • -P specify a directory in which to save a mirrored web site

wget -p -m -k -P ./mirror https://www.gidadel.com # mirror mode: crawl a web site by following links and download the contents of each page found to a specified directory

Installation

brew install wget # macOS

whence#

whence

where#

where

which#

[ w ] which - locate a program file in the user’s path

who#

[ w ] who – display who is on the system

  • who -q Show a brief summary of who is logged in.

  • who -s Show who is logged in along with the line and time fields (without the headers).

whoami#

[ w ] whoami - display effective user id

whois#

[ w ] whois – Internet domain name and network number directory service

write#

[ w ] write – send a message to another user

xargs#

[ w ] xargs – construct argument list(s) and execute utility

  • xargs -0 read input assuming a NUL terminator

xsv#

[ g ] xsv - a command line program for indexing, slicing, analyzing, splitting and joining CSV files

  • brew install xsv

xxd#

[ w ] xxd - make a hexdump or do the reverse

options

  • -b display the file using binary isntead of hex

  • -l print n bytes

  • -s start printing at byte position n

[example]

# start at byte offset 35 and print the next 50 bytes
xxd -s 35 -l 50 somefile.txt

yacc#

[ w ] yacc – parser generator

yes#

[ w ] yes - be repetitively affirmative

zip#

[ w ] zip - package and compress (archive) files

zizzania#

[ g ] zizzania

zoxide#

[ g ] zoxide - a smarter cd command

zcat#

[ w ] zcat, gzip, gunzip - compress or expand files

Converting between numerical bases#

#----------[ DEC -> BIN ]
bc<<<'obase=2;16'     # `10000`

dec=23; pad=8; python -c "print(f'{$dec:0{$pad}b}')"

#----------[ DEC -> OCT ]
printf %o\\n 16       # `20`
bc<<<'obase=8;16'

#----------[ DEC -> HEX ]
printf %x\\n 16       # `10`
bc<<<'obase=16;16'

#----------[ -> DEC ]
echo $(( 2#11111111)) # `255`
echo $(( 8#377))      # `255`
echo $((16#ff))       # `255`

bc<<<'obase=10;ibase= 2;1111' # `15`
bc<<<'obase=10;ibase= 8;17'   # `15`
bc<<<'obase=10;ibase=16;F'    # `15`

printf %d\\n 0xF              # `15`

#----------[ HEX -> BIN ]
hex=F; pad=8; python -c "print(f'{0x$hex:0{$pad}b}')" # `00001111`



man ascii

#----------[ ASCII -> BIN ]
printf  'a' | xxd -b | cut -d' ' -f2 # `01100001`
echo -n 'a' | xxd -b | cut -d' ' -f2 # `01100001`

#----------[ ASCII -> HEX ]
printf  'a' | xxd -p # `61`
echo -n 'a' | xxd -p # `61`

#----------[ DEC -> ASCII ]
printf \\ $(printf %o 97) # `a`
printf \\x$(printf %x 97) # `a`

#----------[ OCT -> ASCII ]
printf \\141              # `a`

#----------[ HEX -> ASCII ]
printf \\x61              # `a`
printf 0x61 | xxd -r      # `a`

URL Coding#

# parameter expansion
#   `$*` expands to all arguments
#   `$_` expands to the last argument of the previous command
#   `//` global replacement

while read ; do
  : "${REPLY//%/\\x}" # replace `+` with ` `
  echo -e "${_//+/ }" # replace `%` with `\x`
done <<< https%3A%2F%2Fgoogle.com%2Fsearch%3Fq%3Durldecode%2Bbash
# https://google.com/search?q=urldecode+bash

# OR

function urldec () {
  : "${*//+/ }"         # replace `+` with ` `
  echo -e "${_//%/\\x}" # replace `%` with `\x`
}
urldec https%3A%2F%2Fgoogle.com%2Fsearch%3Fq%3Durldecode%2Bbash
# https://google.com/search?q=urldecode+bash

var=https%3A%2F%2Fgoogle.com%2Fsearch%3Fq%3Durldecode%2Bbash
echo "$(urldec "$var")"
# https://google.com/search?q=urldecode+bash

# BETTER IDIOM
x=https%3A%2F%2Fgoogle.com%2Fsearch%3Fq%3Durldecode%2Bbash
: "${x//+/ }" ; printf -v y '%b' "${_//%/\\x}" ; echo "$y"
# https://google.com/search?q=urldecode+bash

url='https%3A%2F%2Fgoogle.com%2Fsearch%3Fq%3Durldecode%2Bbash' ; python -c "from urllib.parse import unquote ; print(unquote('$url'))"

Working with strings#

string=abcdefg
while read -n1 char ; do printf %s\\n $char ; done <<< $string

for ((i=1 ; i<=${#string}; i++)) ; do
  echo "${string:"$i"-1:1}"
done

https://stackoverflow.com/questions/29593489/bash-loop-through-all-chars-in-string

Parsing arguments#

Contents