From the Terminal

How to securely share a network mount through an existing SSH server

I recently wanted to share a bunch of files with a friend over the internet. These files were on my mediacenter which is a Windows machine running a RAID. This machine was already sharing these files on a Windows file share but I didn't want to expose these files to the internet. Since my workstation was a Linux box already running SSH server on port 22 with SSH key authentication already working I figured the cleanest thing would be to add a new user and set the home directory to the network share with the shell disabled. Easier said than done so I'm writing this post to discuss some blockers I encountered and how I solved them.

First I needed to add a new user. This is the command I used in Gentoo however in most Linux distros you will likely need to use adduser instead.

sudo useradd ftp -M -N -s /sbin/nologin -R /dev/null
  • -M Don't create a home directory
  • -N No user group
  • -s /sbin/nologin sets the shell to nologin to disable any logins
  • -R /dev/null sets the home directory to /dev/null to disable the home directory

Now we need to edit sshd_config for this specific user.

Near the bottom of the file we add this to the config.

Match User ftp
        AuthorizedKeysFile /etc/ssh/authorized_keys_%u
        ForceCommand internal-sftp
        AllowTCPForwarding no
        X11Forwarding no
        ChrootDirectory /home/ftp

Since the AuthorizedKeysFile depends on having a home directory by default we must override that setting to specify a different one for just this one user. Configure that as you would normally.

For security reasons we disable AllowTCPForwarding so people can't use your SFTP as a free proxy and X11 forwarding so that people can't access your X session. Finally we need to use a chroot directory.

 

My initial instinct was to directly use my media center mount as /mnt/mediacenter/ but this was failing since that directory was a network share. SFTP server requires that the chroot directory is owned by root. So despite us setting this user to not use a home directory I decided to use /home/$user for that reason anyway. But in this case I set it as owned by root instead of the user like normal. Now I had to setup some link to the mediacenter mount. Soft links will not work with SFTP so instead I used the mount command like so.

sudo mount --bind /mnt/mediacenter/Library/Anime/ /home/ftp/Anime

Now we can test it. I decided to use my laptop.

There you have it. A fully secure SFTP over SSH.

This technique should work well with WSL1 and WSL2 as well if you're using Windows.

Getting MacOS style hotkeys working in GNU/Linux

Over the years I've used every operating system. Sometimes all at once. Recently I've taken to using Linux as my primary desktop operating system. Everything I need runs natively on Linux and because I have full control over my environment I never have down time due to forced updates. This is my desktop after all so I'm really switching from Windows where the Control key modifier paradigm is more or less the same as Linux. Still I didn't want to compromise on anything with this new configuration and there was one thing missing from my Macbook that I really wanted.

Of course what I'm talking about is the Command key or more colloquially known simply as the Apple Key and all the shortcuts and hot keys built into macOS that make it such a joy to use.

"Linux lets you change whatever you want."

Is a common thing that people say to bring up a positive aspect of GNU/Linux. Is this actually true though? Take for example my issue at hand: Changing copy/paste hotkeys for all programs. In truth, some tasks are so big in their scope few users would ever be capable of accomplishing them. At least not without a lot of contextual knowledge.

Getting macOS style hotkeys is actually an extremely difficult task in Linux. Especially when you want to avoid faking it by binding some keys to other keys on a per application basis.

Then there's the question of why do it at all?

The truth is I just got tired of accidentally sending SIGINTs when switching to terminal and then accidentally triggering the wrong thing when switching out of terminal. I've just gotten so used to not having to context switch my brain in macOS between programs. It's one of the things that truly makes my Macbook Pro more pleasant to use.

So that's my first reason. Muscle memory.

My second reason is simply that Command/Meta is directly next to the space bar and I just like having a shorter stretching span for my hand.

 

Making it work

For the most part GUI applications written for Linux tend to make a lot of assumptions about what hotkeys can and should be in a very opinionated way. Some types of programs require full customization (like code editors) while others like browsers tend to force certain hotkeys. Desktop managers do provide their own hotkey bindings. For example most KDE based applications will use your settings from KDE's "System Settings" to set hotkeys. Meanwhile in Gnome you can use dbind to set your keybinds.

Of course Cut/Copy/Paste is not enough. It would actually be quite jarring to use CTRL+T for opening a new tab immediately after using Command+V to paste something. Indeed if you go down this path you'll end up having to redefine your hotkeys for well..... everything. It is unfortunate as well that applications these days are extremely hostile to user settings.

Despite the hurdles in front of me I decided to try to make it work.

The first thing I did was swap Left Alt and Left Meta on my keyboard. This was actually pretty easy to do with keyboard mapping configs with the KDE GUI simply known as "System Settings". This was simply to make my normal windows keyboard place the buttons in the positions a real mac keyboard would have. None of this is necessary if you have a real Apple layout keyboard. This setting is only active while in my X session so any alternative TTYs would revert back to my stock layout. If I had a real Apple layout keyboard though this is not necessary at all.

This setting is saved to the text file ~/.config/kxkbrc.

$ cat kxkbrc 
[Layout]
DisplayNames=
LayoutList=us
LayoutLoopCount=-1
Model=pc101
Options=altwin:swap_lalt_lwin
ResetOldOptions=true
ShowFlag=false
ShowLabel=true
ShowLayoutIndicator=true
ShowSingle=false
SwitchMode=Global
Use=true

Now with this setting I am able to fake the same key mapping as a real macOS keyboard would have in my desktop session. So far so good. Now I manually changed all my hotkeys in my code editors, terminal applications, and even in the System Settings app I just mentioned to set global hot keys for all KDE applications. This took some time but was very straight forward for the most part.

I went above and beyond even changing my shortcuts for things like Select All.

While this worked for KDE applications I had to also make this change for GTK+ based programs as well. Luckily this would also cover Chrome, Discord, Elements, and a number of other Electron based programs that I use on a regular basis.

I was lucky that VSCode let me change everything but it was very annoying have to redo all the system based hotkeys like Set cursor to start of the line (in macOS this is Meta+Left Arrow) for each and every place I wanted to use it.

Some websites also use Meta+C for example in Github hitting Meta+C will actually take focus and bring you to the Add Comment UI when in a thread so sometimes on Github.com even with Firefox set to use Meta+C to copy something I am not actually able to use my hotkey because the website overrides it.

With GTK I ended up editing `~/.config/gtk-3.0/gtk.css` and setting these hotkeys.

@binding-set gtk-super-cut-copy-paste
{
        bind "<super>x" { "cut-clipboard" () };
        bind "<super>c" { "copy-clipboard" () };
        bind "<super>v" { "paste-clipboard" () };
        bind "<super>a" { "select-all" (1) };
        bind "<super>z" { "undo" () };
}

* {
        -gtk-key-bindings: gtk-super-cut-copy-paste
}

I might add more but for now this got me to where I wanted with most GTK+ applications. Note that if you plan to run any of programs designed to run as root that use GTK+ such as GParted you should also link or copy this config file to the root account home directory as well.

So in the end it is possible. I was able to achieve what I wanted with all but one program. But it was a huge pain. It's really difficult to figure out what settings do what in a Linux environment even when there is documentation. It's typical for documentation to exist for say GTK+ 2.0 and GTK+ 3.0 but there is no over arching best practice guide for what one should do when they want to cover all programs in an environment.

 

 

It Doesn't Have to Be This Way

The Linux ecosystem can and should do better. The ability to do whatever you want is a double edged sword where even though you can in theory change things, in practice however, sometimes changing things is so onerous that staying sane in the process is a task in itself.

So please if you are a Linux user space developer check the GTK+ 2.0 (~/.gtkrc-2.0), GTK+ 3.0 (~/.config/gtk-3.0/gtk.css) and KDE global shortcut configs (~/.config/kdeglobals) when you probe for the environment during startup so that people can choose their own hotkeys for their desktop environment or simply use the C libraries of each and use the hotkeys present.

Better yet, if you are a developer that already supports various input methods for macOS, Linux, and Windows you can simply provide a toggle for users.

It would be beneficial to Linux Desktop environments to agree to a standard config file for hotkeys allowing users to bind keys as they see fit rather than lurching from one weird keybinding config to the next with no centralization to speak off.

Using Meta instead of Ctrl for Cut, Copy, Paste shortcuts in Firefox for Linux

Set these in about:config

accessibility.typeaheadfind isn't blocked by Meta + C / Meta+V so we need to turn it off.

ui.key.generalAccessKey:   224
ui.key.accelKey: 91
accessibility.typeaheadfind: false
accessibility.typeaheadfind.autostart: false

The case for system wide dependency injection in PHP

So first let me clarify what I mean when I say "system wide dependency injection".

The typical PHP Composer project offers "project-wide" dependency injection. The autoloader is made for your project and you simply require that one file. However, with other languages it's typical for dependencies to be in a shared folder for the entire system.

In composer we could use a "global" directory in global mode as well like this.

The problem is it's only global "for you". So if you decided to run composer global for another user all the packages you have installed would be different. So it's not global. Not even system-wide really since the home directory might not allow read from other users.

So neither of these modes in composer are really system wide.

In other languages system wide dependencies are stored like so.

For example here's Python.

Perl too.

And Ruby.

Okay so what about PHP?

To fill this hole some Linux developers who choose to include PHP code directly in the system have started to build packages for their distro's package managers.

Here we have the install script for symfony-console in portage. You can see the files are actually stored in /usr/share/php. Unfortunately these files don't have automatically generated autoloaders by composer making integrations very messy.

Portage users aren't the only ones.

Clearly some users choose to install PHP code for use system wide.

 

/usr/share/ or /usr/lib

Let's check the Linux Foundation's documentation on the topic.

Even though PHP code isn't architecture dependent, in my opinion, it's more of a library for programming so to that end I think it makes more sense to use /usr/lib or in my case /usr/lib64 just like other scripting languages. It would also avoid the messy businesses of integrating with none composer package managers.

 

I decided to go ahead and experiment with this idea.

I forked composer and made a composer system command. Just like composer global sets the folder to ~/.config/composer/, with the system command it instead sets the folder to /usr/lib64/php or /usr/lib/php if they exist in that order. You must create /usr/lib{64}/php yourself.

The fork of composer is available here: https://github.com/hparadiz/composer

I then loaded in some really awesome PHP libraries.

With this newfound power to no longer have to bundle libs with my scripts I made a database backup utility for this blog in 50 lines of code with SSH tunneling that drops sql.gz to my local home folder.

Feel free to fork it and make your own.

I had the script output to a Dropbox folder since I have it running already. Which is awesome since that's three locations and one of them is managed for me. I'll be hacking at this some more but for now I like this starting place.

I like that this script is fully portable and editable wherever it is. If I compile it to a phar it would be difficult to make changes or loop it for multiple databases without setting up an entire build configuration despite that logic being relatively simple even for a novice.

For me, at least, the reasons to allow composer to manage dependencies at a system wide level instead of having other package managers do it is self evident.

Running Multiple Instances of Discord in Linux

In Linux Discord stores it's data in XDG_CONFIG_HOME which is ~/.config by default. It also uses /tmp for a discord.sock file which it uses to determinate if there is already a running process.

Simply set both TMPDIR and XDG_CONFIG_HOME to an alternative and launch discord.

/bin/bash -c "export XDG_CONFIG_HOME=~/.nya; export TMPDIR=~/nya; /opt/discord/Discord"

In Gentoo Discord installs to /opt so change that to your location.

Solving Dependency Slot Conflicts in Gentoo Elegantly

Sometimes when you want to emerge something or upgrade something you will have dependency slot conflicts like this example.

 

To fix this problem you'd need to emerge all kde-frameworks/* packages you currently have installed to upgrade them all at once since it's all part of one framework.

The following commands let you do so with ease.

equery l kde-frameworks/* -F '$category/$name'

 

The output of which you can send right into emerge like so.

sudo emerge --ask -1 --verbose-conflicts $(equery l kde-frameworks/* -F '$category/$name')

 

Now you'll hopefully get a clean emerge.

Age of Empires II: Definitive Edition on Linux

Get the Linux version of Steam and enable this in settings

 

This will unlock the ability to download and install any title even if they are Windows only titles.

Now you'll be able to enable a custom compatibility tool in the settings for the game.

I've been playing the game with Proton 5.8-GE-2-MF which you can get at https://github.com/GloriousEggroll/proton-ge-custom.

 

You will also need to delete these files to avoid lockups from playing wmv files in-game. Otherwise the game runs flawlessly.

rm -rf ~/.steam/steam/steamapps/common/AoE2DE/resources/_common/movies
rm -rf ~/.steam/steam/steamapps/common/AoE2DE/resources/en/campaign/movies

 

If you are having issues make sure you have the most recent version of DXVK, Proton, Wine and your video card's driver.

Enable Middle Mouse

In Firefox on linux middle mouse click pastes by default. To fix it set these things in about:config

middlemouse.paste: false
general.autoScroll: true

Port forwarding with SSH Tunneling

With OpenSSH, port forwarding is configured using the -L option.

You can initiate a port forwarding operation with this command:

ssh -L 80:example.com:80 technex.us

In this example we are telling OpenSSH to open port 80 on the current machine to example.com on port 80 from the server we are connecting to. In this case the server is technex.us.

Don't forget that anyone can connect to this port on your machine so you might want to limit connects to localhost by telling OpenSSH to listen on a specific IP. In this case you can specify 127.0.0.1 like so:

ssh -L 127.0.0.1:80:example.com:80 technex.us

Since this is OpenSSH you can actually use the alias you specified in your ~/.ssh/config file.

I talk more about the OpenSSH config file here.

Building on top of that guide if you want to maintain a port forward everytime you connect to a specific machine you can use this syntax:

Host alias
	HostName technex.us
	User user
	IdentityFile /Users/user/.ssh/mykey_rsa
	LocalForward 127.0.0.1:80 example.com:80

Now you can simply type in ssh alias in terminal and be connected with a port forward.

Making Any Terminal Command Into a Background Service Daemon

This guide will show you to make a bash script that is self aware about it's location on the file system and therefore able to be used as a global terminal command from any current directory.

It will have the ability to take arguments as commands and have the ability to act on those commands with a set of basic daemon related functions such as start, stop, restart, status, version, and help.

The bash script will also have the ability to save a .pid file, read from the file, and use the file as necessary to send basic signals to the background running process. The script will collect any output from the running process to a log which you can monitor with tail.

First thing's first. Let's create our bash script. Use touch to create a new plaintext file.

user@machine:~# touch servicedaemon

Don't forget to give the file executable permission with this command.

user@machine:~# chmod +x servicedaemon

Now open the file in your favorite text editor and lets start putting our script together. It might be beneficial for you to leave a terminal open so you can play with your script as we go along.

#!/bin/bash

The script must contain this on the first line to tell bash what script interpreter to use. In this case we're using bash of course.

Next we setup a way to handle arguments for our command. In this case we want to show usage information when someone doesn't provide any arguments but also provide basic arguments that are known to most people like -h and -v for help and version information.

#!/bin/bash

# source: https://stackoverflow.com/questions/59895/getting-the-source-directory-of-a-bash-script-from-within
# this detects the real location of the script even if it's linked
SOURCE="${BASH_SOURCE[0]}"
while [ -h "$SOURCE" ]; do # resolve $SOURCE until the file is no longer a symlink
DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
SOURCE="$(readlink "$SOURCE")"
[[ $SOURCE != /* ]] && SOURCE="$DIR/$SOURCE" # if $SOURCE was a relative symlink, we need to resolve it relative to the path where the symlink file was located
done
DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
cd $DIR

me="$(basename "$(test -L "$0" && readlink "$0" || echo "$0")")";

pidfile="$me.pid"

executable="myservice.sh"

function loadenv() {
    if [ -f ".env" ]
    then 
        envtype=$(cat .env)
        envfile="conf/$envtype"

        if [ -f $envfile ]
        then
            echo "Loading environment from $envfile"
            . $envfile
        else
            echo "$envfile not found."
        fi
    else
        echo 'Environment type definition not found. Attempting production.'
        if [ -f 'conf/production' ]
        then
            . conf/production
        else 
            echo 'Production environment definition not found.'
        fi
    fi
}

function status() {
    if [ -f $pidfile ]
    then 
        PID=`cat $pidfile`
        if ps -p $PID > /dev/null
        then
            echo "$me is running on $PID"
        else
            echo "$me is not running"
        fi
    else
        echo 'PID file not found.'
    fi
}

function start() {
    loadenv
    log=$DIR/$me.log

    if [ -f $pidfile ]; then 
        echo "PID file found: " && cat $pidfile && exit 0
    fi

    daemon() {
        echo "Starting $me"
        $executable >> $log 2>&1 &
        pid="$!"
        echo $pid > $pidfile
    }
    daemon
}

function stop() {
    echo 'Stopping $me'
    PID=`cat $pidfile`

    if ps -p $PID > /dev/null
    then
        kill `cat $pidfile`
    fi
    rm $pidfile
}

function version() {
    echo '0.0.1'
    exit
}



function usage() {
    echo "Usage: $me [status|start|reload|stop]" 1>&2;
    exit;
}

case $1 in
    status)
        status
        exit
    ;;
    start)
        start
        exit
    ;;
    reload)
        stop
        start
        exit
    ;;
    stop)
        stop
        exit 
    ;;
    -h|-\?|--help)
        usage
        exit
    ;;
    -v|--version)
        version
        exit
    ;;
    *)
        usage
        exit
    ;;
esac
shift

At this point you can run ./servicedaemon from terminal and see the usage information pop up.

user@machine:~$ ./servicedaemon 
Usage: servicedaemon [status|start|reload|stop]
user@machine:~$ ./servicedaemon status
PID file not found.
user@machine:~$ ./servicedaemon -v
0.0.1
user@machine:~$ ./servicedaemon -h
Usage: servicedaemon [status|start|reload|stop]

Now just replace executable on line 18 with what you want to run. A log of any output will be placed in the same folder with the name of the executable file with a log extension. You can tail it to see output. A PID file will be created in the same folder.

Login to SSH Faster and With Greater Security: The SSH Config File

If you're like me you need to login to multiple servers via SSH on a daily basis. For many years when I was younger I typed in the whole IP or hostname of a server everytime I wanted to login to that server. After learning how to use the ssh config file logging into your SSH machine can be cut down to just a few keystrokes.

The SSH config file is always in ~/.ssh/config

Here's a template you can use.

Host alias
	HostName example.com
	User user
	IdentityFile /Users/user/.ssh/mykey_rsa

You can create as many entrees in the file as you like.

  • Hostname can be a DNS resolved domain name or an IP address but that is what SSH will try to actually connect to.
  • Host is actually just the name of this entry in this case I used "alias".
  • User when you type in the SSH command in terminal you can specify a user like normal but if you don't it will use the option you put in
  • IdentityFile is an optional setting to specify your private key SSH key.

When I type in ssh alias in the terminal it will simply connect to example.com as user.

Here's an example

~/.ssh/config
Host henryparadiz.com hp
        Hostname henryparadiz.com
        User henry
        IdentityFile /Users/henry/.ssh/personal_rsa

Usage

henry@Coder-Laptop:~$ ssh hp
Enter passphrase for key '/Users/henry/.ssh/personal_rsa': 
Welcome to Ubuntu 16.04.4 LTS (GNU/Linux 4.4.0-116-generic x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

23 packages can be updated.
13 updates are security updates.


Last login: Mon Apr 16 05:26:15 2018 from 127.0.0.1

Notice how I use the second alias under the host option simply "hp" which allows me to shorten the entire command to just the above. Easy and simply way to speed up your development.