Etherbox

Context[modifier]

The idea of an "etherbox" arose from experiences developing local-server infrastructures for collaboration for working sessions organized by Constant ASBL. This (as of 2018) still in development project is documented here.


Copy the starting image[modifier]

Downloaded from https://www.raspberrypi.org/downloads/raspbian/

Based on the "lite" image (zip or torrent). As of 20 Jan 2018, this is Raspian "Stretch" lite.

The **lite** image has no desktop / windows session.

Based on 2017-04-10-raspian-jessie-lite.zip

unzip -p 2017-04-10-raspbian-jessie-lite.zip | pv | sudo dd of=/dev/sdc bs=4M

You could use Etcher.io as well

Enable SSH[modifier]

SSH is no longer on by default!

Before putting the SD card in the pi, you can enable ssh. Just create a empty file named "ssh" and save it in the /boot partition of the SD Card.

So mount the SD card and

 cd /media/USERNAME/boot
 touch ssh

Otherwise, you can connect with a screen and run:

sudo raspi-config

Then enable ssh under connectivity.

Find the IP address of the pi[modifier]

Simplest way is to connect on a wired network that has DHCP and plug both the pi and your laptop into a router with ethernet cables, then type:

   ping raspberrypi.local

And you should be able to see the IP address.

Then you can connect with ssh with either:

   ssh pi@raspberrypi.local

or with the IP address in place of "raspberrypi.local" if you are on the wifi.

Login with the default password "raspberry"

Make it easier to login, with an ssh key[modifier]

Starting from your laptop (open a new Terminal session if you are connected to the pi):

   ssh-keygen

Choose the defaults. This generates an "ssh key" pair.

Use the ssh-copy-id utility to send it to the pi.

   ssh-copy-id pi@raspberrypi.local

Bring the rest of the software up to date[modifier]

sudo apt-get update
sudo apt-get upgrade

Change hostname from raspberrypi to etherbox[modifier]

In 2 places:

   sudo nano /etc/hostname
   sudo nano /etc/hosts

change to:

127.0.0.1       localhost
::1             localhost ip6-localhost ip6-loopback
ff02::1         ip6-allnodes
ff02::2         ip6-allrouters

127.0.1.1       etherbox

Best to reboot after this, otherwise it gives warnings all the time.

sudo reboot

Assign Static Ip Adress to the Pi[modifier]

For that, you need to modify a file named 'dhcpcd.conf'. But first, you need to backup this file :

cd /etc/
sudo cp dhcpcd.conf ddhcpcd.conf.original

Then modify the 'dhcpcd.conf' :

 sudo nano dhcpcd.conf

Then find this and replace (with your value) :

# Example static IP configuration:
interface eth0
static ip_address=192.168.1.5/24
#static ip6_address=fd51:42f8:caae:d92e::ff/64
static routers=192.168.1.1
static domain_name_servers=192.168.1.1

Where 192.168.1.5/24 is the new Ip address of your Pi.

At last, reboot your Pi :

sudo reboot

Setup apache to serve the root with custom header + readme's[modifier]

sudo apt-get install apache2
cd /etc/apache2/sites-available
sudo nano 000-default.conf
    ServerAdmin webmaster@localhost
    # DocumentRoot /var/www/html
    DocumentRoot /home/pi/www
    <Directory /home/pi/www>
           Options Indexes FollowSymLinks
           AllowOverride none
           Require all granted
    </Directory>

    HeaderName /include/HEADER.shtml
    ReadmeName README.html

NB: Sets the HeaderName and ReadmeName directives (part of mod_autoindex).

sudo service apache2 reload

droptoupload.cgi[modifier]

Drop to upload is a CGI python script that allows users to drop files to upload them in the apache directory listings.

First enable the cgi-bin with apache

sudo a2enmod cgi
sudo systemctl restart apache2

Download the script to the cgi-bin.

   cd /usr/lib/cgi-bin
   sudo wget https://gitlab.constantvzw.org/aa/etherbox/raw/master/usr/lib/cgi-bin/droptoupload.cgi
   sudo chmod +x droptoupload.cgi


You can test running it with...

./droptoupload.cgi

Like this is just outputs an HTML form. Looking at http://etherbox.local/cgi-bin/droptoupload.cgi should also display an upload form.

The HEADER.shtml (next step) includes a link to this cgi.

/home/pi/include/HEADER.shtml[modifier]

nano /home/pi/include/HEADER.shtml

(don't use sudo)

<script src="/cgi-bin/droptoupload.cgi"></script>
<style>
body {
background: #38b8e9;
color: black;
}
a {
color: white;
}
#logo {
white-space: pre;
font-family: monospace;
}
</style>
<div class="links" style="margin-bottom: 1em">LOCAL:
<a href="/">home</a>
<a href="/" onclick="javascript:event.target.port=9001">etherpad</a>
<a href="/etherdump/index2.html">etherdump</a>
PUBLIC:
<a href="http://erg.be/">erg</a>
<a href="http://erg.activearchives.org/mw/index.php/Etherbox">mode d'emploi</a>
</div>
<style>
.links {
font-family: monospace;
text-transform: uppercase;
</style>
<script>
document.addEventListener("DOMContentLoaded", function () {
  var p = document.querySelectorAll(".top"),
      t = document.getElementsByTagName("table")[0];
  for (var i=0, l=p.length; i<l; i++) {
    document.body.insertBefore(p[i], t);
  }
});
</script>

Better permissions with facl[modifier]

setfacl

sudo apt install acl
sudo addgroup pi www-data

sudo setfacl -Rm g:www-data:rwX /home/pi
sudo setfacl -d -Rm g:www-data:rwX /home/pi

Unfortunately, I had problems then with permissions on the .ssh folder (preventing keys to be used). To remove the fact on just this folder:

sudo chmod g-w /home/pi

Install etherpad[modifier]

And the version of "nodejs" is now 0.10.29~dfsg-2. So let's try it with etherpad...

sudo apt-get install npm git

sudo ln -s /usr/bin/nodejs /usr/bin/node

cd /opt
sudo git clone https://github.com/ether/etherpad-lite.git
sudo mv etherpad-lite etherpad

# TODO: don't create home folder! ... find option
sudo adduser --system --home=/opt/etherpad --group etherpad
sudo chown -R etherpad:etherpad etherpad

Les fichiers se trouvent dans opt/etherpad/var sous forme d'un seul fichier "dirtyDB" - > You should use a dedicated database such as "mysql", if you are planning on using etherpad-in a production environment.

Create your setting file.

cd etherpad
sudo --user etherpad cp settings.json.template settings.json

sudo cp settings.json.template settings.json

then, desactivate the dirty.db file and configure the mysql database

sudo --user etherpad nano settings.json

Find and change this part :

  //The Type of the database. You can choose between dirty, postgres, sqlite and mysql
  //You shouldn't use "dirty" for for anything else than testing or development
 /* "dbType" : "dirty",
  //the database specific settings
  "dbSettings" : {
                   "filename" : "var/dirty.db"
                 },
*/
  //An Example of MySQL Configuration
   "dbType" : "mysql",
   "dbSettings" : {
                    "user"    : "etherpaduser",
                    "host"    : "localhost",
                    "password": "etherpadpass",
                    "database": "etherpad",
                    "charset" : "utf8mb4"
                  },

Set up Mysql[modifier]

First thing to do is to install MySQL :

sudo apt-get install mysql-server

Then create the database, for this we need to login with the Root user (super-user) :

sudo su

Run mysql  :

mysql

Then create the database and the user 'etherpaduser' with the password 'etherpadpass' :

create database etherpad;
grant all on etherpad.* to 'etherpaduser'@'localhost' identified by 'etherpadpass';

Use Ctrl-D to quit mysql. And exit to quit su.

Just to test if it works :

mysql -u etherpaduser -p etherpad

Run etherpad for the first time as the etherpad user...[modifier]

cd /opt/etherpad
sudo --user etherpad bin/run.sh

The first time you run the etherpad software it takes a **long** time as it downloads related packages. It may also give quite some warnings, but hopefully no errors.

Setup etherpad to start as a service[modifier]

sudo nano /etc/systemd/system/etherpad.service
[Unit]
Description=Etherpad-lite, the collaborative editor.
After=syslog.target network.target

[Service]
Type=simple
User=etherpad
Group=etherpad
WorkingDirectory=/opt/etherpad
ExecStart=/usr/bin/nodejs /opt/etherpad/node_modules/ep_etherpad-lite/node/server.js
Restart=always

[Install]
WantedBy=multi-user.target

After this, to start once...

sudo systemctl start etherpad

Check if it's working with:

sudo systemctl status etherpad

And finally automatically start on boot:

sudo systemctl enable etherpad

Change the welcome text

sudo --user etherpad nano settings.json

If you want to keep the pad private, add the __NOPUBLISH__ tag into the body text

etherdump[modifier]

Etherdump is a script that dumps all pads to different format text files. It's run periodically by a cron job to create a running archive of the etherpads.

Install deps:

sudo apt install python3-pip
sudo pip3 install python-dateutil jinja2 html5lib

Install from repo:

cd
mkdir Software
cd Software
git clone http://murtaugh@gitlab.constantvzw.org/aa/etherdump.git
cd etherdump
sudo python3 setup.py install

Init the folder[modifier]

You need the API key. Copy it from:

cat /opt/etherpad/APIKEY.txt

Then...

cd /home/pi
mkdir etherdump
cd etherdump
etherdump init

For the URL use: http://192.168.1.5:9001/

And paste the API key.

Use pandoc to convert pads to HTML[modifier]

Pandoc is used to convert markdown files to HTML.

   sudo apt-get install pandoc

Create an image gallery with imagemagick[modifier]

Imagemagick to make thumbnails.

   sudo apt-get install imagemagick

cron.sh[modifier]

nano /home/pi/cron.sh
# Dump the etherpad to files
cd etherdump
etherdump pull --meta --text --dhtml --pub . --no-raw-ext
etherdump index \
  *.meta.json \
  --templatepath /home/pi/include \
  --template etherdump.template.html \
  --title "Erg etherdump" > index2.html
# Run the makefile
cd /home/pi
make

Make it executable:

chmod +x /home/pi/cron.sh

makefile[modifier]

The makefile defines the "recipe" that turns the pads (when they are named something.md) into HTML. The "%.html: %.md" is an "implicit rule" that defines how any ".md" file (right hand side, or pre-requesite, can be turned into a ".html" file (left hand side, or "target" in the language of the makefile. For this kind of rule the special variables can be really useful (like $< and $@).

This makefile uses the pandoc program to convert markdown to html. This program has LOTS of options is quite powerful. Good documenation on the markdown format, and how pandoc supports it is: http://pandoc.org/MANUAL.html#pandocs-markdown

nano /home/pi/makefile
mdsrc=$(shell ls etherdump/*.md)
htmldest=$(mdsrc:%.md=%.html)

all: $(htmldest)
# $< is the right part input prereq    $@ is the target 
%.html: %.md
	pandoc --from markdown \
		--standalone \
		--section-divs \
		--smart \
		--css styles.css \
		--to html5 \
		$< -o $@

include/etherdump.template.html[modifier]

<!DOCTYPE html>
<html lang="{{language}}">
<head>
	<meta charset="utf-8" />
	<!-- for controlling google robots.txt settings -->
	<meta name="google-site-verification" content="Ro8-A1t6QCIzTm_O49iqKED8YbvVnMELgdKDjy1bnqc" />
	<title>{{title}}</title>
	<link rel="stylesheet" type="text/css" href="{%block css %}styles.css{%endblock%}">
	<link rel="alternate" type="application/rss+xml" href="recentchanges.rss">
	{% block scripts %}
	<script src="jquery-latest.js"></script>
	<script src="jquery.tablesorter.min.js"></script>
	{% endblock scripts %}
</head>
<body>
<h2 id="title">{{title}}</h2>
<p><em>To hide a pad from this listing, use the __NOPUBLISH__ tag.</em></p>
<p>Last updated {{timestamp}}.</p>

<script src="index.js"></script>

<table class="listing tablesorter">
<thead>
<tr>
	<th>name</th>
	<th>versions</th>
	<th>last edited</th>
	<th>revisions</th>
	<th>authors</th>
</tr>
</thead>
<tbody>
{% for pad in pads %}
<tr>
	<td class="name">
		<a href="{{pad.link}}">{{ pad.padid }}</a>
	</td>
	<td class="versions"> 
		{% for v in pad.versions %}<a href="{{v.url}}">{{v.type}}</a> {% endfor %}
		{% if pad.padid.endswith(".md") %}<a href="{{pad.padid.split(".", 1)[0]}}.html">html</a>{% endif %}
	</td>
	<td class="lastedited">{{ pad.lastedited_iso|replace("T", " ") }}</td>
	<td class="revisions">{{ pad.revisions }}</td>
	<td class="authors">{{ pad.author_ids|length }}</td>
</tr>
{% endfor %}
</tbody>
</table>

</body>
</html>

Enable the cron job[modifier]

crontab -e

The following runs the cron.sh every minute.

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin$

# For more information see the manual pages of crontab(5) and cro$
#
# m h  dom mon dow   command
* * * * * /home/pi/cron.sh 2> /home/pi/cron.log

Access point[modifier]

DRAFT --- NOT TESTED

Taken from this "ultimate" guide

apt-get install dnsmasq wireless-tools hostapd

# the next wasn't necessary for jessie, but for completeness..
RPI3 broadcom chip 
apt-get install firmware-brcm80211
rmmod brcmfmac
modprobe brcmfmac

Give fixed IP to wlan0 interface, edit /etc/network/interfaces switch off the built in stuff and add (section 2):

auto eth0
allow-hotplug eth0
iface etho inet dhcp

#################################
# 1. ORIGINAL settings... use wpa_supplicant for client mode
#allow-hotplug wlan0
#iface wlan0 inet manual
#    wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf
#
#################################
# 2. Fixed IP address (for hotspot / hostapd)
auto wlan0
iface wlan0 inet static
address 10.9.8.7
netmask 255.255.255.0
#################################

Replace /etc/dnsmasq.conf with:

interface=wlan0
dhcp-range=10.9.8.10,50.9.8.254,12h
address=/#/10.9.8.7
no-resolv

Edit /etc/hostapd/hostapd.conf file (adjust depending on driver/hardware)

interface=wlan0
driver=nl80211
ssid=WiFeels
hw_mode=g
channel=6

Edit /etc/default/hostapd and add

DAEMON_CONF="/etc/hostapd/hostapd.conf"

Make hostapd start at boot

update-rc.d hostapd defaults

Reboot.

Create a pagekite "backend" (pi)[modifier]

cd /home/pi/Software
wget https://pagekite.net/pk/pagekite.py
chmod +x pagekite.py
sudo nano /etc/systemd/system/pagekite.service
[Unit]
Description=pagekite
After=syslog.target network.target
 
[Service]
Type=simple
User=pi
Group=pi
ExecStart=/home/pi/Software/pagekite.py --clean --frontend=erg2.activearchives.org:10108 --service_on=http://erg2.activearchives.org:localhost:80:lartnesenseignepas
 
[Install]
WantedBy=multi-user.target

Test:

sudo systemctl start pagekite
sudo systemctl status pagekite

Enable:

sudo systemctl enable pagekite

Create a pagekite tunnel frontend (on a public server)[modifier]

See: http://activearchives.org/wiki/Making_a_local_server_public_with_pagekite.py

Configure your laptop's ssh to use the tunnel[modifier]

DRAFT

~/.ssh/config

   Host erg.activearchives.org
   CheckHostIP no
   ProxyCommand /usr/bin/corkscrew %h 10107 %h %p

To shutdown the box cleanly[modifier]

sudo shutdown -h now

To ssh to the pi[modifier]

From your laptop's terminal:

ssh pi@192.168.1.5

And use the password (standard pi password is 'raspberry').

Get a LEGO case for the pi[modifier]

See Pi Blox case

Create a reverse proxy for the etherpad[modifier]

Creating a tunnel to give your local server a public address exposes only the (Apache) webserver (aka traffic on port 80). In order to make the etherpad visible and usable publically, it's possible to create a "reverse proxy" that creates access to the etherpad (on port 9001) via the webserver. To do this we create a special "virtual" location (/pad) and tell apache to redirect traffic to etherpad.

This recipe is based on information found here.

   sudo a2enmod proxy proxy_http headers proxy_wstunnel deflate rewrite


<VirtualHost *:80>
	# The ServerName directive sets the request scheme, hostname and port that
	# the server uses to identify itself. This is used when creating
	# redirection URLs. In the context of virtual hosts, the ServerName
	# specifies what hostname must appear in the request's Host: header to
	# match this virtual host. For the default virtual host (this file) this
	# value is not decisive as it is used as a last resort host regardless.
	# However, you must set it for any further virtual host explicitly.
	#ServerName www.example.com

	ServerAdmin webmaster@localhost
	# DocumentRoot /var/www/html
	DocumentRoot /home/pi
    	<Directory /home/pi>
           Options Indexes FollowSymLinks
           AllowOverride none
           Require all granted
    	</Directory>

    	HeaderName /include/HEADER.shtml
    	ReadmeName README.html

	# Available loglevels: trace8, ..., trace1, debug, info, notice, warn,
	# error, crit, alert, emerg.
	# It is also possible to configure the loglevel for particular
	# modules, e.g.
	#LogLevel info ssl:warn

	ErrorLog ${APACHE_LOG_DIR}/error.log
	CustomLog ${APACHE_LOG_DIR}/access.log combined

	# For most configuration files from conf-available/, which are
	# enabled or disabled at a global level, it is possible to
	# include a line for only one particular virtual host. For example the
	# following line enables the CGI configuration for this host only
	# after it has been globally disabled with "a2disconf".
	#Include conf-available/serve-cgi-bin.conf

    # ETHERPAD PROXY
    ProxyVia On
    ProxyRequests Off
    ProxyPreserveHost on
   <Location /pad/>
        ProxyPass http://localhost:9001/ retry=0 timeout=30
        ProxyPassReverse http://localhost:9001/
    </Location>
    <Location /pad/socket.io>
        # This is needed to handle the websocket transport through the proxy, since
        # etherpad does not use a specific sub-folder, such as /ws/ to handle this kind of traffic.
        # Taken from https://github.com/ether/etherpad-lite/issues/2318#issuecomment-63548542
        # Thanks to beaugunderson for the semantics
        RewriteEngine On
        RewriteCond %{QUERY_STRING} transport=websocket    [NC]
        RewriteRule /(.*) ws://localhost:9001/socket.io/$1 [P,L]
        ProxyPass http://localhost:9001/socket.io retry=0 timeout=30
        ProxyPassReverse http://localhost:9001/socket.io
    </Location>

    <Proxy *>
      Options FollowSymLinks MultiViews
      AllowOverride All
      Order allow,deny
      allow from all
    </Proxy>

</VirtualHost>

# vim: syntax=apache ts=4 sw=4 sts=4 sr noet

Error messages FAQ[modifier]

Failed to restart apache2.service: The name org.freedesktop.PolicyKit1 was not provided by any .service files[modifier]

When you see this you maybe just forgot to sudo, as in:

   sudo systemctl restart apache2
Dernière modification effectuée le 27 juin 2018.