Command line interface tools and tricks

2020-05-09

(last time edited: 2021-10-21)

tags: linux, shell

Ignoring aliases

You can ignore custom aliases by typing \ before the command, just like this:

$ \ls

$ \xrandr

$ \shutdown

$ \rm

This means the program will run without custom parameters you specified in config files such as .bashrc; .mkshrc; etc.

If you find the need to return to the previous directory do:

$ cd -

If you wanna get rid of the annoying stdout echo do this:

$ cd ~-

Using these parameters can help you get to the previous directory instead of going to home directory, it can truly help scripting.

Differences between su and sudo

Back when I started to learn about system administration I had some trouble understanding the differences between su and sudo.

They basically do the same thing but sudo is more flexible with sudoers files and permits the normal user to enter root password in order to perform superuser commands. I still prefer using su.

s stands for substitute and u for user.

Sometimes can be annoying to remember or differentiate if you are in a hurry. Both commands are pretty useful in their own way.

$ su - <user> -c "echo hello world"

$ sudo -u <user> echo hello world

These commands are especifically important if you want to execute a command from a normal user account that's locked with /sbin/nologin as default shell (you can check that out in /etc/passwd) or locked via passwd -l

Easy root login

As su:

$ su root

As sudo:

$ sudo -i or $ sudo su or $ sudo su root

Notice!: Please, do not use sudo! Do not even install it in your system. Don't even use alternatives such as doas. They are not necessary tools.

Lower a vid resolution using FFmpeg

$ ffmpeg -i input_video.mp4 -s 640x360 -c:a copy new_video_output.mp4

Easy way to trim videos using FFmpeg

Sometimes I backup my own Twitch streaming videos. I download them from my video producer dashboard and then cut the parts I like the most.

To simplify this and don't use any kind of GUI video editor program such as olive, avidemux, shotcut, openshot... I just use the glorious ffmpeg.

This way of trimming also saves a lot of time, it does NOT re-encode the video.

$ ffmpeg -i ~/Downloads/somevideo.mp4 -ss 01:14:88 -to 03:11:00 -c:v copy -c:a copy newvideo.mp4

Using the -to parameter will cut out from an specific fixed time to another specific time. This is the best sane way to do it.

If you instead use -t it will extract (in this example) an entire time of 3 hours and 11 minutes after starting from 1h and 14 minutes and 88 seconds. This way of trimming is more random because you'll not know the specific fixed end time you'll get in the new video.

Important! If you don't declare -to it will trim everything until the end of the video. And if you don't declare -ss it will trim from the very start of the video.

Concatenate videos using FFmpeg

List your videos with full path in a new text file like this:

file '/home/username/Downloads/file1.mp4'
file '/home/username/Downloads/file2.mp4'
file '/home/username/Downloads/file3.mp4'

Then just use FFmpeg.

$ ffmpeg -f concat -safe 0 -i listofvideos.txt -c copy newvideo.mp4

Remember all videos need to be encoded with the same format.

List all non-free installed packages in a Void Linux system

I always forget this command.

$ xbps-query -p <repository> -s nonfree

List all available non-free packages in Void Linux repositories

$ xbps-query -Mi -R https://alpha.de.repo.voidlinux.org/current/nonfree -s \*

Check removed-packages in Void Linux

$ xbps-query -p replaces removed-packages

Stream a file using FFmpeg

$ ffmpeg -re -i /path/to/movie.mp4 -c copy -f flv rtmp://server.com/stream?=key=KEYGOESHERE

Simple directory permissions

755 are the default permissions of newly created directories.

# chmod 755 /path/to/dir

or recursively

# find /path/to/dir -type d -exec chmod 755 {} \; -print

Simple file permissions

644 are the default permissions of newly created files.

# chmod 644 /path/to/file

or recursively

# find /path/to/dir -type f -exec chmod 644 {} \; -print

Change PV (Physical volume) name

# dmsetup rename <old-name> <new-name>

Change VG (Volume group) name

# vgchange <old-name> <new-name>

Change LV (Logical volume)

# lvrename <volume-group-here> <old-logical-volume> <new-logical-volume>

Refresh logical volumes

If you get:

WARNING: Device mismatch detected for xxx/xxx which is accessing /dev/mapper/xxx instead of /dev/mapper/zzz.

This conflict happens when logical volumes and dm-crypt mappings are not correctly deactivated. To solve the problem you just simply execute this order:

# vgchange --refresh

Activate a VG

# vgchange -a y <volume-group-here>

Find XBPS packages with less or equal than X dependencies

As a shell function:

findpkg() {

        MAXDEPS=7

        if [ "$*" ]; then
                PACKAGES=$(xbps-query -Rs "$*" | awk '{print $2}' | xargs -n 1 xbps-uhelper getpkgname)
        else
                echo argument required
                kill -INT $$
        fi

        for PKG in $PACKAGES; do

                DEPS=$(xbps-query -Rx "$PKG" | wc -l )
                DESC=$(xbps-query -R -p short_desc "$PKG")

                if [ "$DEPS" -le "$MAXDEPS" ]; then
                        printf '%s %s - %s\n' "$DEPS" "$PKG" "$DESC"
                fi

        done

}

Usage is very simple. Open a virtual terminal and pass any kind of argument.

$ findpkg browser

Record desktop audio using FFmpeg and PulseAudio

$ ffmpeg -f pulse -i default $(date +%s).mp3

Record desktop video with FFmpeg

$ ffmpeg -video_size 1920x1080 -framerate 24 -f x11grab -i :0.0 $(date +%s).mp4

Listing files arguments

I know listing files via terminal can be done by any person, but adding some arguments and trying to remember them can be a pain in the ass, so I save it here instead of creating a useless alias.

$ ls -lahS | less -R

-l for list mode, -a includes hidden files; -h intuitive file sizes, -S sorted by file size.

-R to read raw control characters (ANSI escape code)

Show all processes the easy way

$ ps -a

Show all processes running by a user

$ ps -u <user>

for more info

$ ps -fu <user>

for more info and trees

$ ps -fu <user> --forest

Show user processes with a shell rc function

psu() {
    ps -o user,pid,%cpu,%mem,command -u <user> $1 | grep --color=always -E '[0-9][0-9]\.[0-9]|$' | less -R
}

The variable $1 is important. Takes any kind of argument you pass to it, for example psu --forest.

Show root processes with a shell rc function

psroot() {
    ps -o user,uid,pid,%cpu,%mem,command -u root $1 | grep -v "\[" | grep --color=always -E '[0-9][0-9]\.[0-9]|$' | less -R
}

We can also use this command with the --forest argument.

Print CSV files (comma-separated values file) in terminal

$ cat file.csv | column -ts , | less -S

Find all programs running with exact program name

$ pgrep -xl <program-name>

Allow only specific IPs to connect to your listening port

Replace $IP with the client IP you want to able to connect. Replace $PORT with the port you are listening from.

# iptables -I INPUT \! --src <ip> -m tcp -p tcp --dport <port> -j DROP

Block all UDP ports using iptables

# iptables -A OUTPUT -p udp -j DROP

and

# iptables -A INPUT -p udp -j DROP

Block an attacker IP or specific domains

# iptables -A INPUT -s <ip> -j DROP

if you don't know the IP of a domain get the IP of a domain using getent. Browsers have their own DNS cache so you might want to clear it after adding new iptables rules.

$ getent hosts google.com

Remove all iptables

# iptables --flush

Connect to Twitch's IRC via Irssi

If you suffer from WebSockets disconnection from time to time you can use raw IRC protocol to join Twitch channels.

First get your user's OAuth Password in this generator.

And then connect using Irssi or your favorite IRC client.

The following is using the syntax for Irssi.

/connect -ssl irc.twitch.tv -password oauth:90md2kod90asokdm1o2kdmasd90asd

/join #anychannel

Add taskbar blink to Irssi

/set beep_msg_level MSGS HILIGHT

Resize an image using ImageMagick

$ convert /path/to/image.jpg -resize 400x400 /path/to/image.jpg

or

$ convert /path/to/image.jpg -resize x200 /path/to/image.jpg

Strip exif from image using ImageMagick

# convert /path/to/image.jpg -strip /path/to/image.jpg

You can combine parameters -strip with -resize and more.

Add a user to a group

# usermod -G <group> -a <user>

Remove a user from a group

# gpasswd -d <user> <group>

Get the IP of a domain.

This is a clean and simple alternative to nslookup.

$ getent hosts <domain>

or you can just as well use ping

$ ping <domain>

or just use nslookup.

$ nslookup <domain>

Simple way to list files

$ ls | xargs -0

The output looks very similar to that of using tree but it helps copypasting.

Check size of a directory

$ du -sh /path/to/directory

Move file to another dir and don't rewrite if file exists

$ mv -n file /new/path

The -n argument makes magic here.

Tidy up packages in XBPS

# xbps-pkgdb -m auto <package>

or

# xbps-pkgdb -m manual <package>

Purge orphan packages in Void Linux

# xbps-remove -yo

Print all files of a directory recursively

$ find /path/to/dir -type f -printf "%f\n"

Check type of files using find

$ find /path/to/dir -type f -exec file {} \;

Find duplicate files recursively

Install jdupes.

$ jdupes -dr /path/to/dir

Get a nice calendar

The calendar included in util-linux by default shows 3 months.

The following argument will show you all months of a selected year.

$ cal -y <year>

Replace 2021 with any year.

It's very simple but useful.

Reload gpg-agent

If you wanna refresh the key stored in memory by gpg-agent.

$ echo RELOADAGENT | gpg-connect-agent

Read the gpg-agent current configuration

$ gpg-agent --gpgconf-list

Cool looking network devices information in Json format

$ ip -j -p a

Reset network interface configuration

# ip address flush dev <interface>

Interface could be eth0, wlan0, etc.

Show diff when reading git log commits

$ git log -p

Suspend and wake up automatically

Install util-linux.

Suspend to memory for 5 hours.

-s arguments takes time in seconds

# rtcwake -m mem -s 18000

Suspend and wake up at certain UNIX time.

First get the UNIX time corrected from UTC to localtime.

# TZ=America/Argentina/Salta date +%s -u -d "tomorrow 06:30"

or

# TZ=Europe/Amsterdam date +%s -u -d 15:00

then pass the UNIX time output to rtcwake, for example:

# rtcwake -m mem -t 1623317400

You can save the UNIX time in a variable and pass it to the rtcwake program in the -t argument.

# TZ=America/New_York UNIXTIME2WAKEUP=$(date +%s -u -d "15:00")

and just do something like

# rtcwake -m mem -t $UNIXTIME2WAKEUP

If you don't specify tomorrow, it will take the date as it's for today.

Add service to default OpenRC runlevel

# rc-update add nginx default

Find out which user is using certain processes in certain directories

$ lsof /path/to/dir

Mount everything from fstab

Simple way to mount new devices after fstab changes. -a for all, -v for verbose.

# mount -av

Simple backup with rsync

$ rsync -Pv --delete-after /path/to/RANDOMDIRECTORY /path/to/store/the/dir

In the example above, /path/to/RANDOMDIRECTORY will be placed inside /path/to/store/the/dir

Rsync can be tricky sometimes. Be sure to run --dry-run and not wipe out an entire directory.

-P to show progress.

-v for verbose.

--delete-after to delete files in destination directory after the transfer is done.

-a for archiving. This option executes chown in the remote directory receiving the files. Usually to preserve the original permissions of the transferred files. Sometimes can bring up errors.

OPTIONAL -c for checking checksum, this can slow down transfers but also prevents innecessary copying. Good argument to keep in mind with small files.

Simple creation of a LUKS device + LVM

Create a LUKS big block device out of a partition.

# cryptsetup luksFormat --type luks2 /dev/sdX1

Open LUKS.

# cryptsetup open /dev/sdX1 whatever_name

It is optional but inside the LUKS block device you can create logical volumes or just leave a big filesystem formatted partition with ext4 or some other format.

To create logical volumes you need the LVM2 package. You can create root, home, swap or whatever volume you want.

# pvcreate /dev/mapper/whatever_name

# vgcreate <vg-name> /dev/mapper/whatever_name

# lvcreate -n games -L 90G <vg-name>

# lvcreate -n otherstuff -l 100%FREE <vg-name>

And then format the logical volumes. Example:

mkfs.ext4 -F /dev/mapper/coolvgname-games

mkfs.ntfs -F /dev/mapper/coolvgname-otherstuff

Auto unlock LUKS

Generate random gibberish key and store it inside the LUKS root dir or anywhere you want. It needs to be inside the LUKS device.

# dd bs=1024 count=7 if=/dev/urandom of=/root/encrypteddev.key

Add the key to a new LUKS slot. You'll be prompted to enter the main (slot 1) key.

# cryptsetup luksAddKey "$SECONDARY_DISK"1 /root/encryptedddev.key

Read all slots just in case.

# cryptsetup luksDump /dev/sdX

Change permissions to the key.

# chmod 000 /mnt/nusys/root/encrypteddev.key

Replace LUKS_BLKID_GOES_HERE with the UUID output of:

# blkid /dev/sdX1

Check for Linux modules

# modinfo <module>

# lsmod

Save alsa levels across reboots

Works in all distributions.

# alsactl store

Generating new GRUB configuration

Instead of using wrappers just use the normal GRUB command.

# grub-mkconfig -o /boot/grub/grub.cfg

Fixing EFI boot lists

Remove an entry.

# efibootmgr -b 0004 -B

Load EFI variables

Very useful if planning to install any Linux mini root filesystem from a running Alpine Linux distribution.

# modprobe efivarfs

Listen to specific Pulseaudio input

These commands are very useful if you wanna test your microphone or any kind of 3,5mm jacket input (game or home audio system, etc).

First list all sources.

$ pactl list sources

Load a loopback module. Be sure to replace the source number.

$ pactl load-module module-loopback source=<number>

When you are done and wanna stop listening the loopback just unload the module.

$ pactl unload-module module-loopback

You can achieve the same by automating the selection with multiple UNIX tools. In this example I want to get the source number of my microphone connected to Built-in Audio Analog Stereo (integrated sound card).

$ pactl list sources | grep -B 3 'Built-in Audio Analog Stereo' | head -n 1 | grep -Eo '[[:digit:]]{1,9}'

Remove old unused kernels in Void Linux

# vkpurge rm all

Fix Alpine Linux packages

This command reinstalls all packages.

# apk info | xargs apk fix

Unzip multiple files

Use quotes to make the command work.

$ unzip '*.zip'

Extract 7z files into a directory

The -o argument goes next to the directory output path, it's not a typo.

$ 7z x /path/to/file.7z -o/path/to/dir

Transfer files under certain size

$ rsync -Prv --delete-after --max-size=8388608B *.bin root@192.168.1.105:/mnt/somedisk/directory

-r for recursive otherwise --delete won't work.

Delete files bigger than X size

Delete files bigger than 8 Megabytes. Size goes in bytes.

$ find . -name "*.bin" -type f -size 8388608c -delete

Ask before removal

$ rm -i file

Move files bigger than X size

Move files bigger than 200000 bytes.

$ find . -type f -size +200000c -exec mv "{}" <directory> \;

Check all shell environment profile variables

Simple command but easy to forget.

$ printenv

Print an specific variable.

$ printenv DBUS_SESSION_BUS_ADDRESS

Skip the first line

Useful to manage different columns of piped data.

$ awk '(NR>1) {print $1}' file.txt

$ tail -n +1

Check all UDP and TCP ports listening

$ ss -luta

Check all processes by a user

Similar to ps output but in real time.

$ top -u USER

Check if a virtual terminal has 256 color support

for in {0..255}; do

        printf "\x1b[48;5;%sm%3d\e[0m " "$i" "$i"

        if (( i == 15 )) || (( i > 15 )) && (( (i-15) % 6 == 0 )); then
                printf "\n";
        fi

done

Transform any image into a rgui wallpaper for RetroArch

$ convert original.png -resize x240 -gravity center -extent 320x cropped_and_resized_image.png

$ pngquant --posterize=4 --force -o final_image.png cropped_and_resized_image.png

Find a file and show filename with grep

$ grep -niro SOMETEXT *.txt

Show current established connections to TCP or UDP ports

$ ss -t

$ ss -u

The argument -p brings more info about the process.

Show current established connections to TCP or UDP ports via specific process

For TCP connections.

$ ss -tp | grep <program>

For UDP connections.

$ ss -up | grep <program>

It's also nice to watch established connections in realtime.

$ watch -n1 ss -p

or better:

$ watch -n1 ss -lnt -o "state established '( sport = :445 )'"

Get all current Flatpak package sizes

$ flatpak list --columns=size,name,application,origin

Easily activate any Python virtual environment

These commands can be added as an alias or a function in .bashrc or any shell rc configuration.

$ path="$(find "$(pwd)" -name activate -type f)"

$ . "$path"

Rename a user in UNIX.

This will only change the login name. not the home directory name.

# usermod -l <new-name> <old-name>

Get current local private IP.

$ ip a | grep 192 | awk '{print $2}' | cut -f 1 -d "/"

Reset alsa configuration

Removes all configuration changes made using the amixer cli tool.

$ alsactl restore

Disable Headphone Playback jack

The numid is the playback mixer listed in amixer list.

$ amixer controls

In my case I have 2 for Headphone

$ amixer cset numid=2 off

Check if there is a headphone jack connected

$ amixer cget numid=14

numid is the number for Headphone Jack

if values=off , then there's no jack connected, easy

Change volume for specific numid

$ amixer cset numid=2 30,30

30 for left and 30 for right

Change last Git repository commit message

$ git commit --amend -a

List and show dirs of Pip installed packages

$ pip list -v

Get APK stats in Alpine Linux

$ apk stats

Change volume of a file using ffmpeg

$ ffmpeg -i /path/to/song.ogg -filter:a "volume=0.4" /path/to/newsong.ogg

Update and upgrade in Alpine Linux

# apk -U upgrade

Pretty print json

$ cat <file.json> | python -m json.tool

List Alpine Linux orphan packages

This command is important since all cleaning is done manually.

$ apk list -O

List Alpine Linux packages without info bloat

$ apk list -I | awk '{print $1}' | sort | less

Unzip many files at once

It is necessary to pass single quotes, otherwise it won't work.

$ unzip '*.zip'

Lock accounts for more security

Usually it's safe to lock out accounts with # usermod -p "*" username, it will add a wildcard character in the shadow file for that user. This disables login via password but the account will NOT be locked. You can still log in via SSH keys. You need shadow installed. More info here.

Check if there are accounts with empty passwords and not locked.

# awk -F: '($2 == "") {print}' /etc/shadow

If there are accounts with empty passwords then.

# usermod -p "*" <user>

List files with UID and GID instead of names

-n lists numeric names and group IDs

$ ls -lan

Fix private SSH keys permissions

Usually stored in ~/.ssh

$ chmod 600 <file>

Create a user with specific UID

# useradd -u 1111 -s /bin/mksh -c "computer owner" USERNAME

or

# adduser -u 1111 -s /bin/mksh -g "computer owner" USERNAME

Fix home files and directory permissions

$ chown -hR 1111:1111 /home/USERNAME

Check if user has the correct UID/GID and groups added

$ id <user>

Add a user to many groups

# usermod -aG group1,group2,group3,group4,audio,video,input USERNAME

Print all Environment Variables

$ env

or

$ printenv

Fix Alpine Linux Packages by Downgrading from Edge to Latest-Stable

This command resets all packages to versions available from current repositories.

It's important to run this if previously the system was downloading and extracting packages from the edge repository and now you're using latest-stable repository.

Of course you before need to add latest-stable repository and update+upgrade to refresh the cache.

# apk upgrade -U

and then

# apk upgrade -a

-a is for "available". Which means it will look for the new latest-stable repo we added and will start working with those package versions.

You could as well add -U and do everything in a single command.

# apk upgrade -Ua

There are other commands like

# apk fix

# apk fix -r

# apk fix -u

which are as well very important.

Sort lines in Vim insensitive

With this argument sorting will not be sensitive to upper and lower cases.

Sort lines from line 1 to line 50. The i is important.

:1,50sort i

Disable Internet access for specific user on a system

It is easy to do with a single command line using iptables. The setting won't be saved across reboots.

# iptables -A OUTPUT -m owner --uid-owner <user> -j REJECT

Another alternative route to achieve this same result can be done by creating a separate custom group.

# groupadd nonet

Add the user to the group. This can also be done using usermod.

# adduser <user> nonet or # usermod -aG nonet <user>

Create an iptables rule for disabling Internet to that specific group called nonet.

# iptables -I OUTPUT 1 -m owner --gid-owner nonet -j DROP

Empty a file

$ echo -n > <file>

Get information of your current graphical session

$ loginctl show-session $XDG_SESSION_ID

Quickly create a SSH key

$ ssh-keygen -t ed25519 -f somename -C ""

Better use of dmesg

Make use of less pager to read better with the -H argument.

# dmesg -H

Get Python documentation served via web

This command starts a HTTP server on the port 1234.

$ pydoc -p 1234

Read it in your browser by entering this URL:

http://localhost:1234

Show listening ports

Netstat is part of net-tools.

$ netstat -l

Add -n to show ports instead of names. And -t to only show TCP ports or -u to show UDP ports.

For more info:

$ netstat --help

Fix PulseAudio (Standalone without using PipeWire)

autospawn=false
##.ifexists module-gsettings.so
#.nofail
#load-module module-gsettings
#.fail
#.endif

Polkit

Add your user to wheel group.

# usermod -aG wheel <user>

Login back again to your session to make group changes available.

Now create custom rules under root. To see all possible polkit that actions do: $ pkaction.

These rules usually already come by default.

polkit.addAdminRules(function(action, subject) {
    return ["unix-group:wheel"];
});

Add these ones.

polkit.addRule(function(action, subject) {
    if (subject.isInGroup("wheel")) {
        if (action.id.match("org.freedesktop.login1.reboot") || action.id.match("org.freedesktop.login1.power-off") || action.id.match("org.freedesktop.login1.suspend")) {
            return polkit.Result.YES;
        }
    }
});

Also don't forget to start the elogind service in OpenRC or runit.

Sudoers

Let users in wheel group be able to execute poweroff and reboot binaries without password. (/etc/sudoers.d/wheel)

%wheel ALL=(ALL) NOPASSWD: /sbin/poweroff, /sbin/reboot

Get the Public (External) IP

Install miniupnpc.

$ external-ip

Get information of all forwared ports when using Miniupnpc

$ upnpc -l

Temporarily activate TearFree (vsync on)

$ xrandr --output <output> --set TearFree on

Output could be HDMI-0, HDMI-1 or DVI-D-0. Check that before with:

$ xrandr

Check if it was enabled correctly by doing

$ xrandr --verbose

and look for your display.

Using verbose you can see more details such as the scaling, gamma, brightness, edid, etc.

Check GPU temperature

You can do it without using lm-sensors. Output will be in five-digit format.

$ cat /sys/class/drm/card0/device/hwmon/hwmon*/temp1_input

Check CPU thermal zone temperatures

Same as above.

$ cat /sys/class/thermal/thermal_zone*/temp

Get the list of packages that depend on certain package

$ apk info -r <package>

Read joystick inputs

Install linuxconsoletools. It should contain a binary called jstest.

$ jstest /dev/input/js0

There are other layouts mode you can use to show output. Add these arguments.

--normal, --old, --event, --nonblock, --select

List all joysticks connected and controlled by evdev

Evdev is a generic input event interface in the Linux kernel.

Since version 1.16 the xorg-xserver obtained support for libinput:

kernel → libevdev → libinput → xf86-input-libinput → X server → X client

libevdev is a dependency of xf86-input-libinput

udev will show joystick event devices (evdev) in /dev/input

Install linuxconsoletools.

$ evdev-joystick --l

Get the modified date of a file

$ stat -c %y <file>

Dry run Make

Useful for checking out compiling parameters.

$ make -n <file>

or

$ make --dry-run <file>

For C standard version when compiling

$ gcc -std=c2x -pedantic-errors -o <output> <input>

For warnings add these arguments:

-Wall -Wextra

Check how many files are there in a directory, recursveily

$ find /path/to/dir -type f | wc -l

Check Go (Golang) environment variables

$ go env

Remove exif data from a lot of files

$ exiv2 rm *