Martin Váňa

4 things you might not have known about Git

2018-11-20T08:00:00+00:00

Git is a well known distributed version control system which was designed to handle everything from small to very large projects with speed and efficiency. Many developers (including myself) have used it for years not knowing following things.

Origin of the name

The official readme states:

The name “git” was given by Linus Torvalds when he wrote the very first version. He described the tool as “the stupid content tracker” and the name as(depending on your mood):

random three-letter combination that is pronounceable, and not actually used by any common UNIX command. The fact that it is a mispronunciation of “get” may or may not be relevant.

stupid. contemptible and despicable. simple. Take your pick from the dictionary of slang.

“global information tracker”: you’re in a good mood, and it actually works for you. Angels sing, and a light suddenly fills the room.

“goddamn idiotic truckload of sh*t”: when it breaks

Linus once stated: “I’m an egotistical bastard, and I name all my projects after myself. First ‘Linux’, now ‘git’.”

Git was initially a toolkit for a version control system

Git was initially a toolkit for a version control system rather than a full user-friendly VCS. It has plumbing (low-level commands) and porcelain (high-level commands). Low-level commands allow you to basically create your own VCS. Luckily, that is rarely the case since the porcelain is simply awesome.

SVN interoperability

Git has built-in support for SVN. It allows a developer to take advantage of Git on projects which use SVN, or it can be used to migrating to Git.

Lost data are not really lost

Have you ever accidentally deleted a branch or done a hard reset and later found out you shouldn’t have done that? There is a git reflog which will help you out! It records when the tips of branches and other references were updated in the local repository. For more details see documentation.

Sources

The Fastest and The Safest Web Pages

2018-06-29T07:00:00+00:00

When it comes to the speed and the safety of web pages nothing beats static web pages. However, they are tedious to write and every change across multiple files hurts. That is where static site generator comes in handy.

What is a static site generator?

A static site generator transforms page content written in markup language (such as Markdown) into HTML pages.

Main advantages

Usability — Writing Markdown instead of HTML
Flexibility — Page layout is defined at one place
Speed — Nothing loads faster than a static HTML
Safety — No runtime, no place for malicious code
Free/cheap hosting

Where do I get a static site generator?

There is a list of static site generators at StaticGen.

Which do I choose?

It depends. Personally, I recommend using the one which matches your use case, and/or is written in a programming language you know in case you want to write a plugin.

I can recommend Jekyll and Hugo.

Use cases

This blog uses Jekyll

Blog
Project site
Business / Personal sites
etc.

Hosting

To name a few free hosting services:

What if I need some dynamic content?

It is possible with JAMstack. JAMstack is a modern web development architecture based on client-side JavaScript, reusable APIs, and prebuilt Markup.

Applying Nette Coding Standard with PHP <7.1

2017-08-23T07:00:00+00:00

A few days ago, I came across a How to Apply Nette Coding Standard in Your Project blog post and I fell in love. That was exactly what I was looking for! The only problem is that I do not have PHP 7.1 installed (I have got PHP 7.0) and I cannot update (at least not easily without breaking things).

Solution

Recently, I was fiddling with Vagrant. Therefore, I used gained knowledge to set up a simple vagrant project solving my problem.

Usage

Synchronize project folder

Add project folder, you want to check, to Vagrantfile:

Vagrant.configure("2") do |config|
  ...
  config.vm.synced_folder "path/to/project-to-check", "/vagrant_data/project-to-check"
  ...
end

Note: You can add multiple project folders

Style check & fix

Now run Vagrant and connect via SSH.

$ vagrant up
$ vagrant ssh

Once you are connected via SSH move to /vagrant folder and run check&fix.

$ ./ecs check /vagrant_data/project-to-check/app/ --config vendor/nette/coding-standard/coding-standard-php70.neon --fix

For more details read tool documentation.

For whom is this?

This solution is for people who for some reason do not have PHP ≥7.1 installed and still want to check&fix their PHP coding style according to Nette Coding Standards.

Sources

A cron job using Vagrant

2017-08-01T08:00:00+00:00

In A cron basics we learned how to set up a cron job in a local environment. Today I am going to show you how to set it programmatically using Vagrant.

Provisioning files

A script file

First, we write a script we want to execute. To make it as simple as possible we just write current date to a /vagrant/www/cron.html file

#!/usr/bin/env bash

# File: vagrant/cron/cron-job.sh

# Write current date to cron.html
echo $(date) > /vagrant/www/cron.html

Crontab file

Secondly, we define a crontab file.

# Run example cron job
*/1 * * * * bash /vagrant/vagrant/cron/cron-job.sh

# Comment to make sure there is new line at the end of file
# https://askubuntu.com/questions/23009/why-crontab-scripts-are-not-working/23337#23337

Do you know how often will be this script executed? No? Go check the Online Crontab Editor.

Installation

Finally, we install new crontab file.

#!/usr/bin/env bash

# File: vagrant/cron/cron.sh

echo "----- Provision: Setting up cron ..."
# Overwrite crontab configuration
crontab /vagrant/vagrant/cron/crontab

And we update master unprivileged script because we do not need a privileged user for that.

#!/usr/bin/env bash

# File: vagrant/unprivileged.sh

# ...
# Available unprivileged configurations
bash /vagrant/vagrant/cron/cron.sh
# ...

Does it work?

To see if it works visit http://project.v.martinvana.com/cron.html.

Sources

LAMP with Vagrant

2017-07-28T08:00:00+00:00

In Setting up Vagrant I showed you Hello world project with Vagrant. Now I am going to show you how to set up a LAMP environment.

Provisioning

Provisioning is a way to install and configure software inside a virtual machine. Simplest usable way of doing so is using a shell script. That is what I am going to do.

However, that is not the only option. You can use more advanced automatic configuration and orchestration tools such as Ansible or Puppet if you are proficient with them.

Configuring Vagrant

First, we take a look at Vagrantfile. The minimum configuration contains box definition.

Vagrant.configure("2") do |config|
  config.vm.box = "ubuntu/xenial64"
end

With a configuration like this, our only way of accessing VM is using SSH. Therefore, we create a private network, which allows host-only access to the machine using a specific IP.

Vagrant.configure("2") do |config|
  config.vm.box = "ubuntu/xenial64"
  config.vm.network "private_network", ip: "192.168.33.10"
end

Now we could manually edit /etc/hosts and map 192.168.33.10 address to a hostname project.v.martinvana.com. But life is too short to do things manually.

Therefore we install a vagrant-hostmanager plugin.

$ vagrant plugin install vagrant-hostmanager

Now we can let DHCP select an IP address and we set up a custom IP resolver.

Vagrant.configure("2") do |config|
  config.vm.box = "ubuntu/xenial64"
  config.vm.network "private_network", type: "dhcp"
  config.vm.hostname = "project.v.martinvana.com"

  if Vagrant.has_plugin?('vagrant-hostmanager')
		config.hostmanager.enabled = true
		config.hostmanager.manage_host = true
		config.hostmanager.ip_resolver = proc do
			`vagrant ssh -c "hostname -I"`.split()[1]
		end
	end
end

Since we are using a Virtualbox it is reasonable to limit resources it can consume. Allow using 1GB RAM and 1CPU which can use up to 50% of a single host CPU.

Vagrant.configure("2") do |config|
  # ...
  config.vm.provider "virtualbox" do |vb|
    vb.customize ["modifyvm", :id, "--cpuexecutioncap", "50"]
    vb.memory = 1024
    vb.cpus = 1
  end
  # ...
end

Finally, we got to the provisioning part of a configuration. Folder vagrant in our project is going to contain all provisioning files.

We could define an arbitrary number of provisioning scripts. Therefore I set up three:

install.sh – Install and configure software
load.sh – Actions to do on vagrant up
unprivileged.sh – Actions to do as a unprivileged user

Vagrant.configure("2") do |config|
  # ...
  config.vm.provision :shell, :path => "vagrant/install.sh"
  config.vm.provision :shell, :path => "vagrant/load.sh", run: "always"
  config.vm.provision :shell, :path => "vagrant/unprivileged.sh", privileged: false
  # ...
end

Content of install.sh is following:

#!/usr/bin/env bash

echo "----- Provision: Setting Prague timezone ..."
ln -sf /usr/share/zoneinfo/Europe/Prague /etc/localtime

echo "----- Provision: Add repositories ..."
# ...

echo "----- Provision: Re-synchronize the package index files from their sources ..."
apt-get update

echo "----- Provision: Install the newest versions of all packages currently installed on the system ..."
apt-get upgrade -y

# Available configurations
bash /vagrant/vagrant/apache/apache.sh
bash /vagrant/vagrant/mysql/mysql.sh
bash /vagrant/vagrant/php/php.sh
bash /vagrant/vagrant/utils/utils.sh

# Cleanup
apt-get -y autoremove

As you can see, I am not creating one huge install script but rather several small ones. load.sh and unprivileged.sh files are empty for now.

Apache

To set up Apache we need to configure a virtual host.

# File: vagrant/apache/sites-available/project.conf

<VirtualHost *:80>
  ServerAdmin webmaster@localhost
  DocumentRoot /var/www/www

  LogLevel warn

  ErrorLog ${APACHE_LOG_DIR}/error.log
  CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

# File: vagrant/apache/conf-available/project.conf

<Directory /var/www/>
  Options FollowSymLinks
  AllowOverride All
</Directory>

First, we install Apache, then we create a symbolic link to shared folder /vagrant, and finally, we configure a virtual host.

#!/usr/bin/env bash

# File: vagrant/apache/apache.sh

echo "----- Provision: Installing apache ..."
apt-get install -y apache2

echo "----- Provision: Setup /var/www to point to /vagrant ..."
if ! [ -L "/var/www" ]; then
	rm -rf "/var/www"
	ln -fs "/vagrant" "/var/www"
fi

# Apache / Virtual Host Setup
echo "----- Provision: Install Apache configurations ..."
rm -rf /etc/apache2/sites-enabled/*
if ! [ -L "/etc/apache2/sites-available" ]; then
	if ! [ -L "/etc/apache2/sites-available/project.conf" ]; then
		ln -s "/vagrant/vagrant/apache/sites-available/project.conf" "/etc/apache2/sites-available/project.conf"
	fi
	a2ensite -q project.conf
fi

if ! [ -L "/etc/apache2/conf-available/project.conf" ]; then
	rm -f "/etc/apache2/conf-available/project.conf"
	ln -s "/vagrant/vagrant/apache/conf-available/project.conf" "/etc/apache2/conf-available/project.conf"
fi
a2enconf -q project.conf

To verify that everything went as expected we create a HTML file.

$ echo "Hello world!" > www/index.html

Now we can view the page at http://project.v.martinvana.com/.

The last thing we do is to make sure we restart Apache on vagrant up in order to server configuration file to take effect.

#!/usr/bin/env bash

# File: vagrant/load.sh

echo "----- Provision: Restarting Apache ..."
service apache2 restart

PHP

To install PHP we first have to add PPA with PHP 7.1.

# File: vagrant/install.sh

# ...
echo "----- Provision: Add repositories ..."
add-apt-repository ppa:ondrej/php

# ...

Now we can install PHP, its extensions, and Composer in a similar fashion as in the case of Apache.

#!/usr/bin/env bash

# File: vagrant/php/php.sh

echo "----- Provision: Installing php ..."
apt-get install -y \
        php7.1 \
        php7.1-xdebug \
        php7.1-zip \
        php7.1-mysql \
        libapache2-mod-php7.1

echo "----- Provision: Installing composer ..."
EXPECTED_SIGNATURE=$(wget -q -O - https://composer.github.io/installer.sig)
php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');"
ACTUAL_SIGNATURE=$(php -r "echo hash_file('SHA384', 'composer-setup.php');")

if [ "$EXPECTED_SIGNATURE" != "$ACTUAL_SIGNATURE" ]
then
  >&2 echo 'ERROR: Invalid installer signature'
else
  php composer-setup.php --quiet --install-dir=/usr/local/bin --filename=composer
fi

rm composer-setup.php

To verify that everything went as expected we create a PHP file.

$ echo $'<?php\nphpinfo();' > www/info.php

At http://project.v.martinvana.com/info.php we can view the PHP information page.

MySQL

The only thing we have to take care of during MySQL installation is setting the root password.

#!/usr/bin/env bash

# File: vagrant/mysql/mysql.sh

echo "----- Provision: Installing mysql ..."
# Username: root
# Password: root
debconf-set-selections <<< 'mysql-server mysql-server/root_password password root'
debconf-set-selections <<< 'mysql-server mysql-server/root_password_again password root'

apt-get install -y \
        mysql-server

To verify MySQL (and also Composer) installation I created a simple project with Adminer database management tool.

{
    "name": "vanam/vagrant-kickstarter",
    "type": "project",
    "require": {
        "php": ">=7.1.0",
        "ext-mysql": "*",
        "ext-zip": "*",

        "dg/adminer-custom": "^1.9"
    }
}

Now you should be able to log in as root user.

With composer project in place, we also want to automate dependency updating. For this, we use the unprivileged.sh file.

#!/usr/bin/env bash

echo "----- Provision: Moving to '/vagrant' directory ..."
cd "/vagrant"

echo "----- Provision: Installing composer dependencies ..."
composer install --no-interaction

Pitfalls

You need a stable internet connection during provisioning.
Until you call vagrant up --provision or vagrant provision changes will not be applied.
Provisioning files should be idempotent (have the same effect if run multiple times).
Every configuration change must be written down in a script otherwise it will be lost.
Installation scripts might fail. For advanced workflows use configuration and orchestration tools such as Ansible or Puppet.

Conclusion

As you can see, configuring a first (LAMP) virtual machine is not hard if you have previous knowledge of GNU/Linux systems. There is minimum Vagrant configuration and the rest is just a system configuration.

Full source code is available on Github.

Sources

Setting up Vagrant

2017-07-26T17:00:00+00:00

Did your colleague ever tell you I am unable to run the app on my computer because … or It works on my machine? From now on, this is never going to be a problem again.

What is Vagrant?

Vagrant is a tool for building and managing virtual machine environments. Vagrant is not a virtualization tool. Therefore, you have to get some.

Vagrant comes with out-of-the-box support for Virtualbox which is the most popular, free, and cross-platform. However, it is not the only option. Vagrant also works with VMware, Hyper-V, and Docker.

Advantages

Reproducible and portable environment
Automation
Same environment as production server
All configuration files are plain text files (VCS love it!)
Developer works in his favourite OS, the app runs in a virtual environment
Fast start for a new team member

Installation

It is not recommended to install Vagrant using package managers. Use direct download instead.

Install Virtualbox (or else).

Install Vagrant using a binary package. Debian, Windows, Centos, and Mac OS X systems are supported.

Getting boxes

A box is an image of a virtual machine. You can also create your own box if you desire to do so. However, for most cases, it is a huge speedup for you and your team to utilise already made boxes.

Adding a box

Naming convention of boxes is following: <username>/<box-name>

Let’s say we have chosen the ubuntu/xenial64 box. We could use the box right a way but it is useful to add the box to Vagrant so that multiple Vagrant environments can reuse it (It will take a while to download the box).

$ vagrant box add ubuntu/xenial64

Now we can verify that the box was added.

$ vagrant box list

Initialise a project with Vagrant

First, move to the root of your project. Then execute following:

$ vagrant init ubuntu/xenial64

This will create a Vagrantfile with content:

Vagrant.configure("2") do |config|
  config.vm.box = "ubuntu/xenial64"
end

Surprising but that’s all.

Running a VM and accessing it

Now we are ready to boot the Vagrant environment.

$ vagrant up

There is no UI to access the VM. Therefore, we use the good old SSH.

$ vagrant ssh

How do I transfer files to VM?

Everything in your project folder is automatically synchronised with VM by Vagrant. It appears in /vagrant/ folder.

Let’s try it.

1) Create a new file in your project folder (not in VM)

$ echo "Hello world!" > hello.txt

2) Then go to VM.

$ vagrant ssh

3) List files in /vagrant/ directory.

ubuntu@ubuntu-xenial:~$ ls -la /vagrant/
total 68
drwxrwxr-x  1 ubuntu ubuntu  4096 Jul 26 18:26 .
drwxr-xr-x 24 root   root    4096 Jul 26 18:08 ..
drwxrwxr-x  1 ubuntu ubuntu  4096 Jul 26 18:06 .vagrant
-rw-rw-r--  1 ubuntu ubuntu  3022 Jul 26 17:58 Vagrantfile
-rw-rw-r--  1 ubuntu ubuntu    13 Jul 26 18:26 hello.txt
-rw-------  1 ubuntu ubuntu 46138 Jul 26 18:08 ubuntu-xenial-16.04-cloudimg-console.log

Update your box

The following command will download the new version for your box.

$ vagrant box update

Note that updating the box will not update an already-running Vagrant machine. To reflect the changes in the box, you will have to destroy and bring back up the Vagrant machine.

If you want just to check for updates:

$ vagrant box outdated

Since updating installs new boxes (and leaves the old ones) it is a good practice to remove old boxes from time to time.

$ vagrant box prune

Finishing the work

We can either shut the VM down, suspend or destroy.

Shut down

$ vagrant halt

Halting the virtual machine shuts down the guest operating system and powers down the guest machine. You can use vagrant up when you are ready to boot it again.

Suspend

$ vagrant suspend

Suspending saves the current running state of the machine and stops it. When you are ready to begin working again, just run vagrant up, and it will be resumed from where you left off.

Destroy

$ vagrant destroy

Destroying the virtual machine removes all traces of the guest machine from your system. It’ll stop the guest machine, power it down, and remove all of the guest hard disks.

CLI overview

vagrant box add <username>/<box-name> – Add a box
vagrant box list – List all the boxes that are installed
vagrant box update – Update the box for the current environment if there are updates available
vagrant box outdated – Check if the box you are using in your current environment is outdated.
vagrant box prune – Remove old versions of installed boxes
vagrant up – Create and configure guest machine using your Vagrantfile
vagrant halt – Shut down the running machine
vagrant suspend – Suspend the guest machine
vagrant destroy – Stop the running machine and destroy all resources that were created during the machine creation process.

Conclusion

Setting up a development environment with the virtual machine is a piece of cake with Vagrant. Next time I am going to show you how to set up a LAMP environment.

Sources

A cron basics

2017-07-20T17:00:00+00:00

What is a cron? and How can we use it in web development? are the most common questions people ask. I am going to answer these questions and discuss issues you might encounter.

Glossary

A cron is a programme which executes programmes according to schedule. It is a time-based job scheduler.
A cron task is a programme being executed by cron.
A crontab is a file which contains the schedule of cron tasks to be run and at specified times.

Setting up a cron

Crontab commands

crontab -e – Edit crontab file, or create one if it doesn’t already exist.
crontab -l – Crontab list of cron jobs, display crontab file contents.
crontab -r – Remove your crontab file.

Crontab syntax

*     *     *   *    *        command to be executed
-     -     -   -    -
|     |     |   |    |
|     |     |   |    +----- day of week (0 - 6) (Sunday=0)
|     |     |   +------- month (1 - 12)
|     |     +--------- day of month (1 - 31)
|     +----------- hour (0 - 23)
+------------- min (0 - 59)

Even though the syntax is simple it is much better to use Online Crontab Expression Editor, especially for beginners.

Write a script

It can be basically anything executable on a given system (Bash, Python, PHP, Ruby, C , etc.). It should behave like a normal programme – return zero on success and return non-zero value otherwise.

Use case

Let’s say we are building a simple mailing service [¹]. A task is to send emails to lots of people (~1000) at once eg. newsletter, special offer.

An obvious solution - loop

An obvious solution is to loop through all email addresses and send all emails on the push of the button.

for ($emails in $recipient) {
  $this->sendMail($sender, $recipient, $subject, $content);
}

However, this solution has a flaw – too many emails are being sent during a short period of time. Mail servers do not like that, especially, if you are using the basic ones build in the programming language. It also takes a very long time.

Cron solution

We can create a simple queue of emails using a database:

CREATE TABLE `email` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `sender` varchar(255) COLLATE utf8_unicode_ci NOT NULL,
  `recipient` varchar(255) COLLATE utf8_unicode_ci NOT NULL,
  `subject` longtext COLLATE utf8_unicode_ci NOT NULL,
  `content` longtext COLLATE utf8_unicode_ci NOT NULL,
  `created` datetime NOT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;

Such queue contains mails which are to be sent. Once they are sent, we delete them. Sending script might look like this using Nette and Symfony Console:

<?php

namespace EmailModule\Console;

use Doctrine\ORM\ORMInvalidArgumentException;
use EmailModule\Entity\Email;
use EmailModule\Facade\EmailFacade;
use Nette\InvalidStateException;
use Nette\Mail\Message;
use Nette\Mail\SendmailMailer;
use Symfony\Component\Console\Command\Command;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Output\OutputInterface;

/**
 * Class EmailCommand sends at most 50 oldest emails from email queue.
 *
 * @package EmailModule\Console
 */
class EmailCommand extends Command
{
    /**
     * Command configuration.
     */
    protected function configure()
    {
        $this->setName('app:email')
            ->setDescription('Sends enqueued emails');
    }

    /**
     * Command execution routine.
     *
     * @param InputInterface $input
     * @param OutputInterface $output
     * @return int return status
     */
    protected function execute(InputInterface $input, OutputInterface $output)
    {
        /** @var EmailFacade $emailFacade */
        $emailFacade = $this->getHelper('container')->getByType('EmailModule\Facade\EmailFacade');
        $emails = $emailFacade->getEmailsToSend();

        $sentCount = 0;
        $responseStatus = 0;

        // Try to send all emails
        foreach ($emails as $email) {
            /** @var Email $email */

            $mail = new Message();
            $mail->setFrom($email->getSender());
            $mail->addTo($email->getRecipient());
            $mail->setSubject($email->getSubject());
            $mail->setHtmlBody($email->getContent());

            $mailer = new SendmailMailer;

            try {
                $mailer->send($mail);
                $sentCount++;
                $emailFacade->dequeueEmail($email);
            } catch (InvalidStateException $e) {
                $output->writeLn('<error>' . $e->getMessage() . '</error>');
                $responseStatus = 1;
            } catch (ORMInvalidArgumentException $e) {
                $output->writeLn('<error>' . $e->getMessage() . '</error>');
                $responseStatus = 1;
            }
        }

        $output->writeLn("");
        $output->writeLn($sentCount .  "/" . count($emails). ' emails sent.');

        return $responseStatus;
    }
}

It loops through emails and tries to send them. Now we add a cron task. We want to execute the task with some reasonable period. For example, every 15 minutes send up to 50 emails, that means at most 200 emails per hour (Feel free to modify these numbers to suit your use case). Crontab entry looks like this:

*/15 * * * * /home/www/mail_service/app/console app:email

Pitfalls

Cron task might fail.
Crontab entry might not be set (programmer forgot).
Crontab entry might be set incorrectly (typing error)
Crontab must end with empty line

Conclusion

As you can see, setting up a cron task is simple. First, you write a programme in your favourite language, then you figure out when it should be executed and then you set the cron task up.

Sources

Footnotes

Actually, it is not a very good idea to create a mail sending service like this. To know why read 5 subtle ways you’re using MySQL as a queue, and why it’ll bite you. It is much better to use message queue like RabbitMQ; ↩