Author: Guy

Calculate Google Drive Folder Size Using `rclone`

Google Drive lacks a very basic feature: calculating folder size. There is no solution in the web interface to view the total size of a given directory. There are a couple of dubious looking online “folder size analyzers” request access permissions to your Google Drive and offer you the basic functionality of calculating folder size. While those apps, have a legitimate need for access permission to your account, you may consider given those permissions to random apps a questionable decision.

The (very) useful tool rclone, which provides rsync like interface to many cloud storage providers, implements this functionality. After configuring rclone to work with your Google Drive, use the size command to determine the size of a given folder:

$ rclone size "gdrive:Pictures/"
Total objects: 1421
Total size: 9.780 GBytes (10501374440 Bytes)

The size is calculated recursively. However, there is no simple way to display the size of each sub-directory recursively.

One Line Implementations of GCD and Extended GCD in Python

Sometimes you need quick and dirty implementation of greatest common divisor and its extended variant. Actually, because the fractions module comes gcd function, probably the implementation of the exteded greatest common divisor algorithm is more useful. Both implementations are recursive, but work nonetheless even for comparatively large integers.

# simply returns the gcd
gcd = lambda a,b: gcd(b, a % b) if b else a

# egcd(a,b) returns (d,x,y) where d == a*x + b*y
egcd = lambda a,b: (lambda d,x,y: (d, y, x - (a // b) * y))(*egcd(b, a % b)) if b else (a, 1, 0)

The code above is Python 2 and 3 compatible.

`rsync` and FAT File-Systems

I’m using rsync to sync files from my computer to a FAT formatted SD card. Using the --update flag to rsync makes it “skip files that are newer on the receiver”. It seems that it should work as follows: After the first sync, any subsequent syncs will only transfer those files which changed in the meantime. However, I noticed that transfer times are usually longer than expected, which led me to think that things are not working as they should.

As --update relies only on modification date, obviously something is wrong with it. After ruling out several other possibilities, I’ve guessed that the modification date of some files get mangled. A quick check about FAT revealed that FAT can only save modification times in 2 second granularity. I’ve also used rsync’s --archive flag, which among other things attempts to preserve modifications times of the files it copies. But what about FAT’s lower granularity for modification time? That was apparently the culprit. Some files when copied got a modification time which was up to 1 second before the original modification time! Hence, every time I’ve synced, from rsync’s perspective, the target was older than the source, and hence needs to be overwritten.

This can be demonstrated by the following session:
Continue reading `rsync` and FAT File-Systems

Let’s Encrypt: Reload Nginx after Renewing Certificates

Today I had an incident which caused my webserver to serve expired certificates. My blog relies on Let’s Encrypt for SSL/TLS certificates, which have to be renewed every 3 months. Usually, the cronjob which runs certbot --renew takes care of it automatically. However, there is one step missing, the server must reload the renewed certificates. Most of the time, the server gets reloaded often enough so everything is okay, but today, its been a quite a while since the last time since the nginx server was restarted, so expired certificates were served and the blog became unavailable.

To workaround it, we can make sure nginx reloads it configuration after each successful certificate renewal. The automatic renewal is defined in /etc/cron.d/certbot. The default contents under Debian Jessie are as follows:

# /etc/cron.d/certbot: crontab entries for the certbot package
#
# Upstream recommends attempting renewal twice a day
#
# Eventually, this will be an opportunity to validate certificates
# haven't been revoked, etc.  Renewal will only occur if expiration
# is within 30 days.
SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin

0 */12 * * * root test -x /usr/bin/certbot && perl -e 'sleep int(rand(3600))' && certbot -q renew

The last line makes sure certificate renewal runs twice a day. Append --renew-hook "/etc/init.d/nginx reload" to it, so it looks like this:

0 */12 * * * root test -x /usr/bin/certbot && perl -e 'sleep int(rand(3600))' && certbot -q renew --renew-hook "/etc/init.d/nginx reload"

The --renew-hook runs the next argument after each successful certificate renewal. In our case we use it to reload the nginx configuration, which also reloads the newly renewed certificates.

Update 2019-02-27:

renew-hook has been deprecated in recent versions of certbot. Plus, debian moved from using cronjobs for automatic renewals to systemd timer if they are available. On the other hand, now certbot supports having hooks in configuration files. So, instead of what is described above, i would suggest creating a file /etc/letsencrypt/renewal-hooks/deploy/01-reload-nginx with the following content:

#! /bin/sh
set -e

/etc/init.d/nginx configtest
/etc/init.d/nginx reload

don’t forget to make the file executable.

LyX: Missing title when passing `ignorenonframetext` to Beamer

Passing ignorenonframetext as an option to Beamer, causes it to ignore all the text that is not inside a frame. It is useful when you want to add content for the article version of the presentation (or simply script lines for yourself) that would not show in the regular presentation. LyX puts the title elements outside any frame. Therefore, if you use ignorenonframetext you end up missing the title frame. The solution is to manually wrap the title block (the title, author, institute, etc.) in a frame and append to it \maketitle. This will cause the title frame to be rendered correctly.

Optimize LaTeX PDF Output for Kindle

Kindle can display PDFs, but usually the result is very hard to read. Normal PDFs are not suitable, especially when it comes to paper size for the, relatively, small display of the Kindle. For a forthcoming project, which I intend to write in LaTeX and read on Kindle, I looked into optimizing the document settings so the result will be rendered in a readable manner on Kindle.

I’ve started with the normal article class. The result, is not good at all:
article While Kindle zooms-in automatically to remove, the usually very wide, margins LaTeX uses, the big (A4) paper size, still results in a tiny font on the Kindle display. Switching to Koma-Script, is a bit better, but mainly provides better mechanisms to control the paper size for later experiments.
scrartcl

The next try, is simply to use the A5 paper size. The result is getting better, but the paper size is still too big. Setting the paper size manually to 12cm by 9cm (the screen’s physical dimensions) and setting the pagestyle to empty (removes the page numbering among other things) results in a much better results because of the (still) wide margins and the auto-zoom in a font size that is too big and not enough content fits in a page:
scrartcl_12x9

Finally, by manually setting the text area to be a bit smaller (11cm by 8cm) than the paper size, results in small margins and very little auto-zoom. The output can be clearly read on the Kindle, and still quite a bit of text fits on a single page:


The LaTeX code for the last example is:

\documentclass[DIV=calc,paper=9cm:12cm,pagesize]{scrartcl}

\areaset{8cm}{11cm}
\pagestyle{empty}
\usepackage{lipsum}
\begin{document}
\lipsum
\end{document}

KOMA-Script: Specifying Binding Correction for RTL Documents

The KOMA-Script bundle provides an option to specify the amount of binding correction needed in order to compensate for the width lost in the binding process. By default, it is added to the left margin, which is where the binding is applied for Left to Right languages. However, if a document is written in Hebrew or Arabic, one binds it on the right. The KOMA-Script manual does not consider that option. After a bit of playing I’ve found out that simply using a negative value for the binding correction works.

For example, if in an English document you would use

\documentclass[BCOR=8.25mm]{scrreprt}

For Hebrew you would set

\documentclass[BCOR=-8.25mm]{scrreprt}

Installing OrderNet Pro on Debian Jessie

OrderNet is a a popular stock trading platform in Israel. OrderNet comes in two versions: The regular version is based on Silverlight and can be used on Linux using Pipelight. The Pro version is a desktop program written in .NET Framework and features better interface. This post walks through the steps needed to get OrderNet Pro running on Debian Jessie using Wine.

Continue reading Installing OrderNet Pro on Debian Jessie