Linux

Namespace

This is the basis of docker. It allows you to simulate a network on your machine.

ip netns
ip netns add <NAME>
ip netns exec <NAME> <COMMAND> # execute a command in another namespace

Mount/Unmount

following commands must be run as root

fdisk -l # view the disk and the partition

/dev/sd?  -> physical disk
/dev/sda? -> partitions on the sda disk

Mounting on /mnt/data

mkdir -p /mnt/data # create an empty dir
parted -l # determine the filesystem type
mount -t <ext4> /dev/sd /mnt/data

Unmounting

The argument is either the mount point or the name of the disk

umount /dev/sdb
umount /mnt/data

Shell redirection

/dev/null # empty, import anything, it will disappear
/dev/zero # endless zeros
/dev/urandom # endless random numbers
/dev/full # file that always returns the error code ENOSPC (No space left on device)

# following ones are links to the kernel file descriptor
/dev/stdin # standard input stream 			# 0
/dev/stdout # standard output stream		# 1
/dev/stderr # standard error output stream  # 2

Redirects

By default, it redirects stdout.

• > is an overwrite method
• >> is an append method

Following lines are equivalent

echo "Hello, World!" > file.txt
echo "Hello, World!" 1> file.txt

• >& is the syntax for redirecting the stream to a file descriptor.

# redirecting stderr to stdout
ls file-that-does-not-exist 2>&1 # if no '&' the file named "1" will be created

Redirect stdout and stderr to the same place

ls -la /tmp /dir-that-does-not-exist 1>file.txt 2>&1
cat file.txt

Command line tips

Self-signed certificate

openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -sha256 -days 3650 -nodes

awk

From timestamp to human readable date

TZ=Ameria/Los_Angeles awk '/path to look for/' { print strftime ("%T", $1), $3, $7} file.log

Creating large files from your terminal

The dd command will create a sparse file.

dd if=/dev/urandom of=your-filename bs=2G count=1
# bs = block size (1M = 1024Mb)
# count = number of blocks

The fallocate command will not create a sparse file, which means it is much faster.

fallocate -l 1G your-filename

Traffic control - TC

tc must be run as root

qdiscs (queuing discipline - modify the scheduler) buffer between the protocol stack and the network interface. By default, it uses a FIFO approach.

filter determines which classful qdisc a packet should be enqueued to. (Can qualify the packet based on things like: source/destination/IP...)

classes a class is a sub-qdisc. A class may contains another class. Using classes, we can configure the QoS in more detail.

# delay to the egress (outgoing packets) scheduler
tc qdisc add dev eth0 root netem delay 200ms
# delay = the network property to modify
# netem = network emulator (emulate a WAN property)

Example

$ ping -c 4 google.com
PING google.com ..... 56(84) bytes of data.
64 bytes from ..... : icmp_seq=1 ttl=116 time=9.86 ms
64 bytes from ..... : icmp_seq=2 ttl=116 time=12.2 ms
64 bytes from ..... : icmp_seq=3 ttl=116 time=11.3 ms
64 bytes from ..... : icmp_seq=4 ttl=116 time=10.5 ms

--- google.com ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 8ms
rtt min/avg/max/mdev = 9.859/10.971/12.185/0.870 ms
$ tc qdisc add dev eth0 root netem delay 200ms # add some delay
$ ping -c 4 google.com
PING google.com ..... 56(84) bytes of data.
64 bytes from ..... : icmp_seq=1 ttl=116 time=210 ms
64 bytes from ..... : icmp_seq=2 ttl=116 time=210 ms
64 bytes from ..... : icmp_seq=3 ttl=116 time=211 ms
64 bytes from ..... : icmp_seq=4 ttl=116 time=210 ms

--- google.com ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 7ms
rtt min/avg/max/mdev = 209.870/210.072/210.585/0.546 ms

tc qdisc del dev eth0 root # delete all rules
# instead of del or add, possible values are: show (see what are default rules), change
tc disc change dev eth0 root netem delay 200ms 10ms # +/- 10ms uniform distribution
# instead of delay, possible values are: 'loss 10%' (packet loss of 10%), corrupt, duplicate.

Bandwidth limit

tc qdisc add dev eth0 root tbf rate 1mbit burst 32kbit latency 400ms
# tbf: token buffer to manipulate traffic rates
# rate: sustained max rate
# burst: max allowed burst
# latency: packets with higher latency get dropped


$ iperf -c speedtest.serverius.net -p 5002
------------------------------------------------------------
Client connecting to speedtest.serverius.net, TCP port 5002
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[  3] local 192.168.1.15 port 60442 connected with 178.21.16.76 port 5002
write failed: Broken pipe
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0- 0.1 sec  84.8 KBytes  12.6 Mbits/sec
$ sudo tc qdisc add dev eth0 root tbf rate 1mbit burst 32kbit latency 400ms
$ iperf -c speedtest.serverius.net p 5002
------------------------------------------------------------
Client connecting to speedtest.serverius.net, TCP port 5002
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[  3] local 192.168.1.15 port 43834 connected with 178.21.16.76 port 5002
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.3 sec  1.38 MBytes  1.12 Mbits/sec

Exiftool

Rename images to their creation/modification date attributes

exiftool -ext jpg '-FileName<CreateDate' -d %Y%m%d_%H%M%S%%-c.%%e current_filename.jpg
# -d: specify a date format Y=year m=month d=day H=hours M=minutes S=seconds
# %-c: add a counter if multiple images have the same name.
# %%e: keep the extension

exiftool -ext jpg '-FileName<filemodifydate' -d %Y%m%d_%H%M%S%%-c.%%e ./some-directory
# it will execute the command on all images inside the directory

iperf

Tool used to measure the maximum achievable bandwidth on IP networks. More on this: iperf3.

# server side
iperf --server
# client side
iperf --client HOST_IP

iptables

Here are some useful commands for the iptables firewall:

List rules

iptables -L -v
# -v: verbose mode

Reset rules

# Start with empty tables
iptables --flush [chain] # delete all rules in chain or all chains
iptables --delete-chain [chain] # delete a user-defined chain or all chains

Block all incoming connection

iptables -P INPUT DROP

IP black listing

iptables -A INPUT -s IP -j DROP
# -A: append a rule to the INPUT chain (incoming traffic)

# you can log all incoming traffic from specific IP
iptables -A INPUT -s IP -j LOG --log-prefix "Blocked IP: "

# deleting a rule blocking traffic from an IP address
iptables -D INPUT -s IP -j DROP
# -D: delete

Accept connection to a specific port

# Accept connection on port 22 using the TCP protocol
iptables -A INPUT -p tcp --dport 22 -j ACCEPT

# if you don't need that port anymore
iptables -D INPUT -p tcp --dport 22 -j ACCEPT

Accept connection from specific range of IPs

sudo iptables -A INPUT -p tcp --dport 80 -s 192.168.1.0/24 -j ACCEPT

Save current rules

iptables-save > file.txt

Restore rules

iptables-restore < file.txt

DROP vs REJECT

• use REJECT when you want the other end to know the port is closed (e.g. a ping would get a Destination Port Unreachable error. This indicates the destination was reached but no echo reply was sent back).
• use DROP for connections to hosts you don't want people to see (e.g. a ping would not get any response at all. The request will time out).

SOCAT - TCP/TLS Proxy

The following commands will redirect traffic incoming on port 7000 to a remote host (127.0.0.1:4444).

TCP Proxy

socat -d TCP-LISTEN:7000,fork,reuseaddr TCP4:127.0.0.1:4444

TLS Proxy

socat openssl-listen:7000,fork,reuseaddr,verify=0,key=./key.pem,cert=./cert.pem openssl-connect:127.0.0.1:4444,verify=0

Wget - make offline mirror of a site

wget --mirror --convert-links --adjust-extension --page-requisites --no-parent http://example.org
# shorter way
wget -mkEpnp http://example.org

• --mirror – Makes (among other things) the download recursive.
• --convert-links – convert all the links (also to stuff like CSS stylesheets) to relative, so it will be suitable for offline viewing.
• --adjust-extension – Adds suitable extensions to filenames (html or css) depending on their content-type.
• --page-requisites – Download things like CSS style-sheets and images required to properly display the page offline.
• --no-parent – When recursing do not ascend to the parent directory. It useful for restricting the download to only a portion of the site.

losetup

Allows files to be mounted as block-type devices (hard drives). This makes it possible to create encrypted containers or test file systems.

# creation of a 10 MB file that will serve as a disk
dd if=/dev/zero of=/tmp/disk.dd bs=1M count=10

# attach this file as if it were a block device
losetup /dev/loop0 /tmp/disk.dd # appears with lsblk

# format the disk (cryptsetup to encrypt its contents)
mkfs-ext4 /dev/loop0

# Un/mount disk
mount /dev/loop0 /mnt
unmount /dev/loop0

# detach the file
losetup -d /dev/loop0

# delete disk
rm /tmp/disk.dd

chroot

changes the root directory for the currently running process and its descendants.

chroot jail with only bash & ls (careful it is aliases)

mkdir -p jail/bin jail/lib64/x86_64-linux-gnu jail/lib

cp $(which ls) jail/bin
cp $(which bash) jail/bin

# you'll need bash & ls dependencies
ldd $(which ls) # cp outputs where it needs to go
ldd $(which bash) # cp outputs where it needs to go

sudo chroot jail bash
# test commands
$ ls && exit

mknod

to create a device file (usually in the /dev branch, but not necessarily)

• c => character device
• b => block device
• p => fifo

MAJOR/MINOR:

• 5 => zero
• 7 => full
• 8 => random

# recreate /dev/full
sudo mknod /tmp/full c 1 7
sudo chmod 666 /tmp/full
echo "hello" > /tmp/full

# create a pipe file
mknod /tmp/pipe p
cat /tmp/pipe

echo "hello" >  /tmp/pipe