Small python trick: Better way to init dictionary

From my php and perl background I remember that named arrays or hashes should be initialize correctly, like this:

if (haskey(x, somehash)) {
  somehash[x] += y;
} else {
  somehash[x] = y;
}

And it looks like a tumour inside a python code. But .get() method can cure it:

somedict[x] = somedict.get(x, 0) + y

Looks much better! 

XSendfile usage in Nginx

Suppose you have a corporate site or an internal portal or something else with access from web, but it's not a public site. And to gain access to any page or file your customer at least must be authenticated. It's not a problem for any web app, but it's a problem for static files (pdf, docx, xlsx and so on). With direct link anyone can download some sensitive information. And there are a lot of ways to share such links with the world unintentionally (mail, IM, web spiders, etc).

Old school way to protect a site from downloading direct links is to use http basic auth. It can solve a problem but It also creates some new problems. You need to manage one more database with users logins and passwords for basic auth and sync it with your app. Also http basic auth in web server is week against brute force attack. You can't use many tricks against brute force which are available for you in app. And don't forget about all your users pain because of double authentication process.

There is more elegant way to solve a problem with direct links. Nginx has XSendfile feature. You can use it for static files protection in such way:

1. Nginx works as a frontend for your App.
2. All links to static files send to App.
3. If App thinks that user is authenticated it sends response with X-Accel-Redirect header with real path to static file, otherwise it sends response with some http 4xx error.
4. In nginx vhost location for static files are marked with internal directive which protect files in it from direct download without X-Accel-Redirect header from upstream server.

Short example for Django App:

url(r'^app/static/(?P.*)$', StaticFile.as_view(), name="static_file")
 
class StaticFile(View):
    def get(self, request, path=None):
        if not request.user.is_authenticated():
            return HttpResponseForbidden("Access denied")
        if not path or not os.path.isfile(os.path.join(settings.STATIC_URL, path)):
            return HttpResponseNotFound("Not found")
        mimetypes.init()
        try:
            if "." in path:
                mimetype = mimetypes.types_map[".%s" % path.split(".")[-1]]
            else:
                raise KeyError
        except KeyError:
            mimetype = "text/plain"
        response = HttpResponse()
        response['X-Accel-Redirect'] = "/static/%s" % path
        response['Content-Type'] = mimetype
        return response

And location in nginx:

location /static/ {
    internal;
    root /www/app/;
}

Nginx + youtrack + systemd

Out task is to deploy YouTrack on CentOS 7.X behind Nginx.

What we need to be done for that task:

1. Download and install rpm with latest java from java.com
2. Download YouTrack jar file
3. Add new user for youtrack process and home dir for files
4. Create youtrack.service file for systemd for youtrack service management
5. Configure host in nginx 

1 and 2 need no additional comments. Only one thing, I placed youtrack jar file in /usr/local/sbin and created symlink to current version:

~ la -l /usr/local/sbin/
total 115508
-rwxr-xr-x 1 root root 118280108 Dec 9 13:42 youtrack-6.5.16953.jar
lrwxrwxrwx 1 root root 22 Dec 17 12:41 youtrack.jar -> youtrack-6.5.16953.jar

From management and security point of view it's better to create new user for new service:

useradd -m -d /opt/youtrack youtrack

Now we need .service file for youtrack in systemd:

~  cat /etc/systemd/system/youtrack.service
; /etc/systemd/system/youtrack.service
[Unit]
Description=JetBrains Youtrack
After=network.target
After=syslog.target

[Install]
WantedBy=multi-user.target
Alias=youtrack.target

[Service]
User=youtrack
Group=youtrack
PermissionsStartOnly=true
ExecStartPre=/usr/bin/mkdir -p /var/run/youtrack
ExecStartPre=/usr/bin/chown -R youtrack:youtrack /var/run/youtrack/
PIDFile=/var/run/youtrack/main.pid
ExecStart=/usr/bin/java -Xmx1g -Djava.security.egd=/dev/zrandom -Djava.awt.headless=true -Duser.home=/opt/youtrack -Djetbrains.youtrack.disableBrowser=true -jar /usr/local/sbin/youtrack.jar 127.0.0.1:4080
ExecStop=/bin/kill -HUP $MAINPID

[Install]
WantedBy=multi-user.target

Two things are important here. Since /var/run is in tempfs in CentOS 7.X you need to create pid file dir before service starting up:  

PermissionsStartOnly=true
ExecStartPre=/usr/bin/mkdir -p /var/run/youtrack
ExecStartPre=/usr/bin/chown -R youtrack:youtrack /var/run/youtrack/

And second one is youtrack's start command in:

ExecStart=/usr/bin/java -Xmx1g -Djava.security.egd=/dev/zrandom -Djava.awt.headless=true -Duser.home=/opt/youtrack -Djetbrains.youtrack.disableBrowser=true -jar /usr/local/sbin/youtrack.jar 127.0.0.1:4080

Last part is nginx config:

server {
    server_name youtrack.domain.com;
    listen 80;
    charset utf8;

    rewrite ^ https://$host$request_uri? permanent;
}

server {
    server_name youtrack.domain.com;
    listen 443 ssl;
    charset utf8;

    root /www/youtrack.domain.com/htdocs; 

    ssl on;
    ssl_certificate             /etc/nginx/ssl/youtrack.domain.com.crt;
    ssl_certificate_key         /etc/nginx/ssl/youtrack.domain.com.key;
    ssl_session_timeout         5m;
    ssl_protocols               TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers                 ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv3:+EXP;
    ssl_prefer_server_ciphers   on;

    location / {
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-Server $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto https;
        proxy_pass http://localhost:4080/;
        client_max_body_size 64M;
    }

    access_log /var/log/nginx/youtrack.domain.comu/access.log main;
    error_log /var/log/nginx/youtrack.domain.com/error.log warn;
}