Ivo Petkov

Responsively Lazy v3 is even better for lazy loading images

Sat, 30 Dec 2023 21:58:07 +0000

There is a library called Responsively Lazy I’ve released in 2015 that got quite popular.

Since then a lot has changed. The browsers got the tag, srcset is now widely supported, and the loading attribute was introduced. But still, I think Responsively Lazy is the best way to go when you want to lazy load responsive images and other content.

Version 3.0, released today, builds on the foundation (the idea from 2015) and provides a lot more possibilities.

Let’s check it out.

What’s not changed

Responsively Lazy still provides the same SEO-friendly way to lazy load images. You can see it in Google Images. All websites at Alle.bg and Bear CMS benefit from this. And still, it’s truly lazy. There are absolutely no unnecessary requests.

What’s new

There is a new API. Now you must use the data-responsively-lazy attributes instead of a class name.

Here is an example that contains all the new features.

As you can see, there is now support for AVIF and you can specify the order the image versions are checked.

You can also lazy load background images.

You can customize the load threshold.

All good stuff.

Here is a demo.

Take a look at the code and give it a try.

Сайт на годината 2020 и моите впечатления като участник и част от журито

Mon, 07 Dec 2020 12:25:19 +0000

Тази година имах възможността да се докосна до конкурса Сайт на година и да погледна зад завесите. С тази статия искам да ви споделя моите впечатления и ще се радвам, ако успея да ви мотивирам да участвате през следващите издания, защото ползи има много.

Какво е Сайт на годината?

Това е конкурс, който дава възможност на всеки собственик или създател на уебсайт да покаже това, което е направил през изминалата година и да се състезава за приза “Сайт на година” в една от деветте категории. Жури, коeто включва експерти по програмиране, маркетинг и дизайн, оценява проектите и определя победителите и техните подгласници. Освен това, публиката също има възможност да избере своите победители, чрез онлайн гласуване.

Подготовката

Конкурсът премина през няколко етапа, като естествено през първия етап участниците регистрират своите проекти. През това време жури, което се състоя от над 40 професионалиста определи и публикува критериите за оценка. Аз бях част от екипа програмисти. Заедно с колегите, направихме няколко срещи и публикувахме следните критерии за оценка на техническата част на уебсайтове. Можете да погледнете и критериите за оценка в сегмента Маркетинг и сегмента Дизайн.

Не мога да пропусна да спомена и организационните срещи с Жюстин Томс, която е ръководителят на екипа и главният виновник това събитие да се случва за трети пореден път. Във всяка среща виждах много ентусиазъм. Всеки искаше това издание да бъде по-добро от предишните и да се даде максимална стойност на участниците. Това успя да ме “зарази” и когато дойде моментът за гласуване, бях готов да бъда обективен и максимално полезен.

Гласуването

Гласуването на журито беше разделено на две части. През първия етап, всеки член на журито оценяваше всички сайтове от една или няколко категории (фирмен сайт, личен сайт, сайт за кауза, продуктов сайт и т.н.). Важно условие е да не оценяваш категории, в които има проекти, с който си обвързан. На мен ми се паднаха категориите “Сайт на кауза” и “Фирмен сайт”. За всеки уебсайт трябваше да дам оценка от 1 до 10, и да оставя своя коментар.

Разполагахме с 14 дни и се радвам, че започнах с преглеждането на проектите сравнително рано. Бързо осъзнах, че техническият анализ и писането на коментари (обратна връзка към автора на сайта) може да отнеме много време. Със сигурност отнема много енергия, особено като се има предвид, че си бях обещал да подходя отговорно и да дам максимална добавена стойност.

Веднага след изтичането на този етап и сумирането на оценките, стана ясно кои са топ сайтовете във всяка категория и дойде моментът всеки от нас да избере своите топ 3. Тук отново трябваше да отделя доста време за да оценя техническата реализация на най-добрите уебсайтове от другите категории. Естествено подходих професионално и единственото нещо, което остана е да изчакам церемонията по награждаването, на която всички щяхме да научим финалните резултати.

Награждаването

Тази година награждаването беше онлайн. Можете да го видите във Facebook страницата на конкурса. Можете да видите и списък със самите победители на bgweb.bg. Победителите получиха своите статуетки, но може би по-важното, е че всички получиха обратна връзка и много полезни коментари от едни от най-добрите програмисти, дизайнери и маркетолози в България. Ето това за мен е безценно! Ето това вдига нивото на българския уеб!

Моето впечатление като участник

Конкурсът ме хвана в момент, в който все ще разработвах новия си проект in stealth mode. Реших да го регистрирам веднага и се надявах това да ме мотивира да успея да го завърша по-бързо. Успях да го пусна публично преди крайния срок за регистрация, но уви, не беше на нивото, на което исках да бъде. Това ми даде възможност да “предизвикам” колегите от журито и да видя техните коментари и оценки. Очаквано, коментарите бяха много полезни, а оценките бяха напълно заслужени.

Надявам се, че тази статия ви е била полезна и интересна, и ще се радвам, ако съм успял да ви "запаля" да участвате през следващата година. Дотогава имате няколко месеца, през които да дадете най-доброто от себе си и да създадете страхотни уебсайтове и мобилни приложения. Нека “затрудним” журито, като му дадем да избира само между топ проекти!

И ако има дни, в които не сте мотивирани, не намирате смисъл или ви е трудно, конкурсът “Сайт на година” може да е това, което ще ви издърпа напред, защото можете да разчитате, че вашият труд няма да остане незабелязан.

How I improved the performance of my big PHP web project

Sun, 02 Jun 2019 13:18:15 +0000

In this article, I'll talk about the single architecture approach that improved the most the performance of Bear CMS (it's a content management system I work on) and made me confident about its future updates. I have to admit that designing and optimizing systems is more like a passion for me than just a job, but I can also see the benefits this brings when running such systems on a high scale (CPU cycles can be costly and user time is too valuable to waste).

You may have heard about lazy loading

This is a concept that enables resources to be loaded and initialized only when needed. It's recommended when rendering a web page that has a large number of images (an online shop for example) and most of them are under the fold (not visible until the user scrolls). You may have heard of autoloading PHP classes using spl_autoload_register(). Composer already uses this technique, so you can be certain no precious memory and time is wasted when using third-party libraries.

Lazy loading requirements

To apply this concept you need the following components:

1. A way to register a resource for lazy loading.

2. A mechanism to detect if a resource is needed and load it.

The spl_autoload_register() method from the example above, with a combination of a function (to include the classes) covers both requirements.

How lazy loading helped me big time

I hope you will agree, but a Content Management System is no small project. There is a very diverse set of features that must be supported (themes, editing UI, pages, content elements, etc.) and many different types of requests to handle (HTML pages, images, form submits, background tasks, etc.). To make development easy and enable future improvements without performance implications I decided to enforce lazy loading in the heaviest APIs. Here is an example:

$cms->themes->register('the-name-of-the-theme', function() {
    // initialize the theme here.
});

This enables registering multiple themes and only using the one needed, only when needed.

As you can see the trick here is using ...

Anonymous functions!

Anonymous functions in PHP enable you to postpone or skip the execution of some code. Even defining an array takes memory, and it makes a difference if this array holds lots of data. Let's compare:

$data = array_fill(0, 1000000, 'John'); // Fills an array with a million text entries

The code above takes around 35MB memory and 0.016 seconds on my machine.

$source = function() {
    return array_fill(0, 1000000, 'John');
};

This code takes 0 memory and 0 seconds because ... the content of the function is not executed. Smart! :)

This type of code (the code that is never run in a request) is the single biggest reason I was able to lower the CPU and memory requirements of my application and still keep it functional.

The real code from the CMS example above looks more like this:

$cms->themes->register('the-name-of-the-theme', function($theme) {
    $theme->options = 'data!'; // Defines theme options so that the administrators can customize it
    $theme->styles = 'data!'; // Defines ready-made styles to choose from
    $theme->manifest = 'data!'; // Information about the theme and its author
});

As you can see there are a lot of things that need to be done to define a theme. They all take memory and CPU cycles and are not needed on every request. Requests for images, favicons or sitemaps do not need themes.

Anonymous functions as properties

We can take the approach even further and bring anonymous functions to properties. Here is another update on the themes example above:

$cms->themes->register('the-name-of-the-theme', function($theme) {
    $theme->manifest = function() {
        return [
            'name' => 'My awesome theme',
            'description' => 'This is ...',
            'author' => [
                'name' => 'John Smith',
                'email' => 'example@example.com'
            ]
        ];
    };
});

As you might have guessed it, the information in the manifest is needed only in very rare cases, so there is no need to spend time and memory on every request.

Identifying code that can be skipped can make a big difference.

Anonymous functions are beautiful!

If you've written, or seen JavaScript code, you might take them for granted as they are the basic way of defining asynchronous code (in setTimeout for example). In PHP however, due to lack of async support, they are rarely used. You might have used them to sort and filter arrays and I'd like to encourage you to use them more. They keep your code clean (and can even improve it) and can benefit the performance significantly. Even if you are not architecting systems or working on big projects, please, challenge yourself to try them for recursions or other internal optimizations.

Thanks

Thanks for getting this far. I hope this article has inspired you to optimize your projects and improve their performance. I'll be happy to learn about your results and thoughts on this technique.

Reading locked files in PHP

Fri, 22 Dec 2017 13:20:47 +0000

I hope that you are already familiar with the awesome file locking mechanism that PHP and your OS provide. If not, here is a ...

Brief introduction to file locking

File locking is a mechanism that allows you to prevent read and write access to a specific file for a specific time. You request a write lock on a file, and if it is provided to you (it can be denied too), you can be sure that no other application/instance/process can write to the same file until you release the lock.

Why use file locking

The main reason is that you do not want invalid/broken data. Imagine you use a file to store some JSON data. In one moment you want to read the data, modify it and then write it back. You will be surprised to find out that at the exact same moment (milliseconds apart of course) some other application is doing the same thing and overwriting your modifications. In such case, you should be using file locking.

File locking in PHP

PHP provides a very simple, yet effective mechanism to acquire and release locks. Here is an example:

$pointer = fopen("file.json", "c+"); // open the file for writing
if (flock($pointer, LOCK_EX)) { // acquire a writer lock
    // do some data manipulation
    flock($pointer, LOCK_UN); // release the lock
} else {
    // the file cannot be locked
}
fclose($pointer);

If a file is locked (for reading or writing) in one application, all other applications that request locks on the same file will be paused until the lock is released in the first application. This is the default blocking mode, and it works great.

Now we are ready to continue with the main topic of this post ...

What happens when you read a locked file

In the following lines, I will show you few ways that you can use to read a file while it is locked for writing (and modified) and the results you will get. Also, for a bonus, I will add the result of the filesize() function.

Reading a locked file with file_get_contents()

This is one of the worst way to read a file while it is locked and modified, because:

- file_get_contents() will return an empty string (like in "")

- filesize() will return the actual number of bytes written to the file

$content = file_get_contents('file.json'); // will return empty string
clearstatcache('file.json'); // clear the file cache for the next function
$size = filesize('file.json'); // will return the actual file size in the moment

Reading a locked file with fread() without a reading lock

This is the other worst way to read a file while it is locked and modified, because:

- fread() will return an empty string (like in "")

- filesize() will return the actual number of bytes written to the file

$pointer = fopen('file.json', 'r');
$content = fread($pointer, 1000); // will return empty string
clearstatcache('file.json'); // clear the file cache for the next function
$size = filesize('file.json'); // will return the actual file size in the moment
fclose($pointer);

Reading a locked file with fread() with a reading lock (shared lock)

This is the option to pick if you are OK waiting for the write lock to be released. PHP automatically pauses the execution of the application, so you do not have to do anything special. Just keep in mind the max_execution_time option in your PHP config.

$pointer = fopen('file.json', 'r');
if(flock($pointer, LOCK_SH)){ // will block execution until the write lock is released
    $content = fread($pointer, 1000); // will return the correct content
    clearstatcache('file.json'); // clear the file cache for the next function
    $size = filesize('file.json'); // will return the correct size
}
fclose($pointer);

Reading a locked file with fread() with a reading, nonblocking lock

This is the advanced option. You can request a lock while the file is locked and modified in another application, but instead of waiting for file lock release, the execution will continue.

$pointer = fopen('file.json', 'r');
$wouldblock = null; // a variable that will be set to 1 if the file is already locked
if (flock($pointer, LOCK_SH | LOCK_NB, $wouldblock)) { // LOCK_NB tells flock() to continue execution of the application
    $content = fread($pointer, 1000); // will return the correct content
    clearstatcache('file.json'); // clear the file cache for the next function
    $size = filesize('file.json'); // will return the correct size
} else {
    if ($wouldblock) {
        // the file is currently locked by another application but we can do something else
    } else {
        // cannot acquire lock for other reason
    }
}
fclose($pointer);

Final thoughts

As you can see file locking is not a scary thing. It is quite useful if done properly. I will be happy for you to take the following from this post:

Always lock your files (when reading and writing) if multiple applications/instances/processes have write access to those files.

One more thing ...

I would like to recommend you a library of mine that brings locking benefits to more complex tasks - tasks that you do not want to interfere with one another. The library is simply called Lock, and you can find it at GitHub at https://github.com/ivopetkov/lock. Its based on file locking and works great in apps that share a common filesystem.

Advanced PHP: 7 useful techniques that are rarely used

Tue, 31 May 2016 08:08:40 +0000

In human languages, not all words are equally used, but they all have a purpose and unique power. The same is true for programming languages. A big part of the code we write is just variables, functions, classes, and loops. They are part of the basics you need to know to get your application done. Today I am not going to talk about them. Instead, I'll show you some of the PHP techniques I rarely use but have great impact.

Let's get started.

Anonymous functions and classes

Sometimes in your code, you must provide a callback function (in usort for example). Instead of defining a global function, you can create one dynamically, use it once or twice and remove it (if you want). It can also save you some time thinking about the right name (because naming things is important).

In the following example an anonymous function is used as a callback to help sort people by age.

$list = [
    ['name' => 'John', 'age' => 31],
    ['name' => 'Mike', 'age' => 25],
    ['name' => 'Ben', 'age' => 27]
];
usort($list, function($a, $b) {
    if ($a['age'] == $b['age']) {
        return 0;
    }
    return ($a['age'] < $b['age']) ? -1 : 1;
});

Next, a function is dynamically created and saved into a variable, then called and finally destroyed.

$sqrt = function($a) {
    return $a * $a;
};
echo $sqrt(2);
unset($sqrt);

PHP 7 adds support for anonymous classes, so you can get creative with that too.

Autoloading classes

It's a best practice to create one file per class when writing object-oriented applications in PHP. That way the classes are easier to develop and maintain and the application can load only the classes needed to complete the request. This can be achieved by registering a class autoload function that will include the class file when needed. So instead of writing:

include "some/dir/Class1.php"
include "some/dir/Class2.php"
include "some/dir/Class3.php"

you can write

spl_autoload_register(function ($class) {
    include "some/dir/" . $class . ".php";
});

And then the following code will work just fine.

$object1 = new Class1();
$object2 = new Class2();
// Class3 will not be loaded because it's not needed

Convert errors to exceptions

Prior PHP 7 error reporting in PHP was a bit messy. In PHP 7 there are some improvements. Converting errors to exceptions technique has served me well for years, so would like to show it to you. It's a couple of lines and will improve the way you manage errors.

set_error_handler(function($errorNumber, $errorMessage, $errorFile, $errorLine) {
    throw new \ErrorException($errorMessage, 0, $errorNumber, $errorFile, $errorLine);
});

So now the only thing you have to worry about is catching exceptions. That's a lot easier than suppressing errors, registering custom handlers and taking care of different error handling configurations (in php.ini, runtime, custom handlers and so on). Here is an example:

try {
    echo 5 / 0;
} catch (Exception $e) {
    print_r($e);
}

Magic methods

Object-oriented programming is very popular these days, and PHP provides a great way to bring it to the next level. Magic methods allow you to tweak calls to methods or properties of a class and update the object state when particular operation occur.

Imagine you have to create the following class:

class Person
{
    public $age = null;
    public $eyesColor = null;
    public $hairColor = null;
}

You can provide the magic method __construct to set the default values for the object created.

class Person
{
    public $age = null;
    public $eyesColor = null;
    public $hairColor = null;
    function __construct(){
        $this->age = 20;
        $this->eyesColor = 'blue';
        $this->hairColor = 'brown';
    }
}

You can validate properties using __set and retrieve them using __get.

class Person
{
    private $data = [];
    public $eyesColor = null;
    public $hairColor = null;
    function __set($name, $value)
    {
        if ($name === 'age') {
            if (is_int($value) && $value >= 18) {
                $this->data[$name] = $value;
            } else {
                throw new InvalidArgumentException('Age is invalid. Must be at least 18.');
            }
        }
    }
    function __get($name)
    {
        return isset($this->data[$name]) ? $this->data[$name] : null;
    }
    function __isset($name)
    {
        return isset($this->data[$name]);
    }
    function __unset($name)
    {
        if (isset($this->data[$name])) {
            unset($this->data[$name]);
        }
    }
}

See the full list of magic methods at php.net.

Xdebug

Debugging and profiling are a requirement when working on large applications and for years, there has been a tool called Xdebug that does this in a wonderful way. So now you can easily find the slow places in your code and fix them. Head over to xdebug.org for more information.

register_shutdown_function()

As the name implies this function registers a function that will be called when the execution of the request is about to finish. I often use it to check for fatal errors and print friendly output.

register_shutdown_function(function() {
    $errorData = error_get_last();
    if (is_array($errorData)) {
        ob_end_clean();
        echo 'Error occured! - ' . $errorData['message'];
    }
});

Another great use case is to log something.

Command line

PHP is mostly known as one the popular server languages that power our web pages. But it's not just for web pages. You can write useful script and programs that can be called from the command line. Here is an example:

php hello-world.php -name John

And this is the code of the program. The variable $argv contains all the arguments passed. Index 0 is the filename.

if (isset($argv[1]) && $argv[1] === '-help') {
    echo 'Enter -name  so I can greet you properly.';
    exit();
}
if (isset($argv[1], $argv[2]) && $argv[1] === '-name') {
    echo 'Hello, ' . $argv[2];
    exit();
}
echo 'Invalid command. Type -help for help.';
exit();

Thanks

I hope these techniques will be useful for you too.

All the code from the examples is available at my GitHub repo.

Install PHP and Apache from source

Wed, 04 May 2016 12:57:58 +0000

Installing software in Linux using a package manager is easy as typing "yum install the-name-of-the-thing". But sometimes you want to make deep customizations or make multiple installations of the same software (different PHP or Apache httpd versions for example). Then the only thing you can do is install the software from its source, and this is not that easy. In the following lines you will learn how to install PHP (with PHP-FPM and OPcache) and Apache from source code.

Installation steps

The steps are the same for both PHP and Apache httpd.

Download the archive containing the source and extract it to some directory
Install dependencies (if any)
Configure the installer (enable or disable features)
Run the installer
Configure the software

What to expect at the end

After executing the following commands, you will have a fresh PHP installation, a running PHP-FPM daemon and an Apache HTTP server that will be executing PHP files with the help from the PHP-FPM daemon (Apache will proxy PHP files to PHP-FPM).

Prerequisites

Compiling PHP and Apache httpd requires a compiler. Let's install it.

yum install gcc

Let's create a user that will be used by Apache and PHP and the dir where our files will be located

mkdir /var/www
groupadd www-group
useradd -d /var/www -g www-group -s /bin/false www-user
chown www-user:www-group /var/www

Create the dir where the source will be downloaded and extracted (it may already exists).

mkdir /usr/local/src/

Installing PHP from source

Create the dir where PHP will be installed

mkdir /opt/php-7.0.4

Download the PHP source and extract it to the target directory

wget http://us.php.net/get/php-7.0.4.tar.bz2/from/this/mirror -O /usr/local/src/php-7.0.4.tar.bz2
tar jxf /usr/local/src/php-7.0.4.tar.bz2  -C /usr/local/src

Install some dependencies

yum install libxml2 libxml2-devel

Change the current directory

cd /usr/local/src/php-7.0.4

Configure the installer

./configure --prefix=/opt/php-7.0.4 --enable-fpm --enable-opcache

Build and install the software

make && make install

Create the configuration files (or copy the recommended ones) to their proper places

cp /usr/local/src/php-7.0.4/php.ini-production /opt/php-7.0.4/lib/php.ini
cp /opt/php-7.0.4/etc/php-fpm.conf.default /opt/php-7.0.4/etc/php-fpm.conf

In the php.ini file (located at /opt/php-7.0.4/lib/php.ini) add the following code

zend_extension=/opt/php-7.0.4/lib/php/extensions/no-debug-non-zts-20151012/opcache.so
opcache.enable = 1
opcache.enable_cli = 1
opcache.memory_consumption = 128
opcache.interned_strings_buffer = 8
opcache.max_accelerated_files = 10000
opcache.use_cwd = 0
opcache.validate_timestamps = 0
opcache.save_comments = 0
opcache.load_comments = 0
opcache.enable_file_override = 1

In the php-fpm.conf file (located at /opt/php-7.0.4/etc/php-fpm.conf) make some changes.

Remove the following line (it should be last in the file)

include=/opt/php-7.0.4/etc/php-fpm.d/*.conf

Add the following lines

pid = run/php-fpm.pid
[www]
user = www-user
group = www-group
listen = 127.0.0.1:8999
pm = dynamic
pm.max_children = 10
pm.start_servers = 2
pm.min_spare_servers = 2
pm.max_spare_servers = 4

Start PHP-FPM

/opt/php-7.0.4/sbin/php-fpm --fpm-config /opt/php-7.0.4/etc/php-fpm.conf

You can stop it later with the following command

pkill php-fpm

It's done! Now we have a PHP installation and a running PHP-FPM daemon. Let's install Apache httpd and point it towards PHP-FPM.

Installing Apache httpd from source

Create the dir where Apache will be installed

mkdir /opt/httpd-2.4.20

Download the Apache source and extract it to the target directory

wget http://apache.cbox.biz//httpd/httpd-2.4.20.tar.bz2 -O /usr/local/src/httpd-2.4.20.tar.bz2
tar jxf /usr/local/src/httpd-2.4.20.tar.bz2 -C /usr/local/src

Download some dependencies

wget http://apache.cbox.biz//apr/apr-1.5.2.tar.bz2 -O /usr/local/src/httpd-2.4.20/srclib/apr-1.5.2.tar.bz2
tar jxf /usr/local/src/httpd-2.4.20/srclib/apr-1.5.2.tar.bz2 -C /usr/local/src/httpd-2.4.20/srclib
mv /usr/local/src/httpd-2.4.20/srclib/apr-1.5.2 /usr/local/src/httpd-2.4.20/srclib/apr
wget http://apache.cbox.biz//apr/apr-util-1.5.4.tar.bz2 -O /usr/local/src/httpd-2.4.20/srclib/apr-util-1.5.4.tar.bz2
tar jxf /usr/local/src/httpd-2.4.20/srclib/apr-util-1.5.4.tar.bz2 -C /usr/local/src/httpd-2.4.20/srclib
mv /usr/local/src/httpd-2.4.20/srclib/apr-util-1.5.4 /usr/local/src/httpd-2.4.20/srclib/apr-util
yum install pcre-devel

Change the current directory

cd /usr/local/src/httpd-2.4.20/

Configure the installer

./configure --prefix=/opt/httpd-2.4.20 --enable-so

Build and install the software

make && make install

In the httpd.conf file (located at /opt/httpd-2.4.20/conf/httpd.conf) make the following changes

Enable (uncomment) these modules

LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_fcgi_module modules/mod_proxy_fcgi.so

Change the user and the group

User www-user
Group www-group

Add the following line at the end

ProxyPassMatch ^/(.*\.php(/.*)?)$ fcgi://127.0.0.1:8999/var/www/$1

Start Apache

/opt/httpd-2.4.20/bin/apachectl start

You can stop it later by running

/opt/httpd-2.4.20/bin/apachectl stop

Create a test file to test the installations

echo "> /var/www/info.php

Done! We now have running Apache and PHP.

A better HTML5 parser for PHP

Thu, 31 Mar 2016 15:22:47 +0000

HTML and PHP have existed for a long time, and one of the main use cases for PHP is to render HTML. Unfortunately, it is not that easy to parse and modify HTML in PHP, especially HTML5. That's why I created a simple library that will help you do that, and more, easily. It extends PHP's native DOMDocument library, so it is very familiar to use. DOMDocument is a very powerful library, but it doesn't work quite well with HTML5. So, meet HTML5DOMDocument - an open source library that extends DOMDocument, fixes some issues and adds some functionality.

The fixes

Preserves white spaces - the DOMDocument library removes some white spaces between text and HTML tags, and sometimes they are important.

Preserves - the DOMDocument library convertes to space character (" ").

Preserves void tags - the DOMDocument library converts tag to , which is an invalid HTML tag.

The new

Inserting HTML - Sometimes you need to insert dynamically HTML snippet in an HTML code. The most common case is to append it to the bottom. You may also want to insert a whole HTML document. Then the head content will be added in the proper place. You can also specify an insert target for the body content. Here is an example:

$dom = new IvoPetkov\HTML5DOMDocument();
$dom->loadHTML('Hello ');
// Find the div element and appends insert target
$dom->querySelector('div')->appendChild($dom->createInsertTarget('target1'));
// Inserts the HTML snipped into the insert target
$dom->insertHTML('world', 'target1');
echo $dom->saveHTML();
// Output will be: Hello world

Querying the DOM - The method querySelectorAll is very popular in the JavaScript world only because it's very helpful, and now it's available for DOMDocuments in PHP. Here is an example:

$dom = new IvoPetkov\HTML5DOMDocument();
$dom->loadHTML('Div 1
Div 2');
$divElements = $dom->querySelectorAll('div');
// $divElements will be a DOMNodeList

I hope you'll find this library helpful. Download and contribute at GitHub.

Let's Encrypt on EC2

Mon, 29 Feb 2016 10:04:08 +0000

The year 2016 started with a couple of free SSL certificate solutions. CloudFlare offers a free certificate for all their customers; Amazon joined with AWS Certificate Manager, and Let's Encrypt is in public beta. Today I want to show you how easy it is to get a free certificate from Let's Encrypt and automatically renew it in the future. I'll provide the commands for an Amazon Linux AMI, but they are similar for other Linux distributions.

There are some things you should know about Let's Encrypt:

Certificates last only 90 days.
You are "forced" to automate the process. Actually, this is great, and I'll show you how to do it.
They do not offer wildcard certificates.

Requirements

An email address. You may get useful information about your certificates.
The domain pointing to a directory on the server, that's accessible on the Internet. Let's Encrypt servers will access a file on http://example.com/some_secret_file_name to validate that you own the domain.
Clearly understand that Let's Encrypt is currently in beta, and you will be running a beta version software on your server.

The commands

Install some requirements for the following steps.

yum install python27-devel git

Clone the letsencrypt repository and run the installer.

git clone https://github.com/letsencrypt/letsencrypt /opt/letsencrypt
/opt/letsencrypt/letsencrypt-auto --debug

Create a config file that will be used for new certificates and renewals. It contains the private key size and your email address.

echo "rsa-key-size = 4096" >> /etc/letsencrypt/config.ini
echo "email = email@example.com" >> /etc/letsencrypt/config.ini

Request a certificate for your domain and it's www subdomain. You must also specify the root directory of the domain.

/opt/letsencrypt/letsencrypt-auto certonly --webroot -w /var/www/yourdomainroot -d yourdomain.com -d www.yourdomain.com --config /etc/letsencrypt/config.ini --agree-tos

Remove the directory that was used for validation. This step is optional.

rmdir /var/www/yourdomainroot/.well-known

The certificates are located at /etc/letsencrypt/live/ and the last thing is to update your webserver's configuration. For apache it will look like this:

Listen 443

    ServerName yourdomain.com
    DocumentRoot "/var/www/yourdomainroot"
    SSLEngine on
    SSLCertificateFile /etc/letsencrypt/live/yourdomain.com/cert.pem
    SSLCertificateKeyFile /etc/letsencrypt/live/yourdomain.com/privkey.pem
    SSLCertificateChainFile /etc/letsencrypt/live/yourdomain.com/chain.pem
    SSLProtocol All -SSLv2 -SSLv3
    SSLHonorCipherOrder on
    SSLCipherSuite "EECDH+ECDSA+AESGCM EECDH+aRSA+AESGCM EECDH+ECDSA+SHA384 EECDH+ECDSA+SHA256 EECDH+aRSA+SHA384 EECDH+aRSA+SHA256 EECDH+aRSA+RC4 EECDH EDH+aRSA !RC4 !aNULL !eNULL !LOW !3DES !MD5 !EXP !PSK !SRP !DSS"

Be sure to add the renew command in a crontab. Refresing your webserver command should also be here.

/opt/letsencrypt/letsencrypt-auto renew --config /etc/letsencrypt/config.ini --agree-tos && apachectl graceful

That's it. I hope it was helpful.

Object Storage

Sun, 31 Jan 2016 15:58:05 +0000

I've always been a fan of simplicity. And the simplest method to store data, and the first one that comes into mind when you are a newbie in programming, is the file system. The file API (read, write, delete, etc.) is simple, and the access is pretty fast (especially on SSDs). Also, you can easily list files, sync them and archive them. It's just beautiful and works for huge files too.

A couple of years ago, cloud services like AWS S3 and Google Cloud Storage emerged. They offered the same simple API. The files are called "objects" and are identified by keys, rather than directory names and file names. The object can have metadata, can be versioned and automatically deleted, and can be stored on multiple machines without changing the API. The only missing thing is searching for objects (the only way to search is by prefix).

I wanted all these features in my PHP apps locally and in production. And I wanted them available through a simple API. Simple API allows improvements in the backend to be made in the future. One of them could be using Memcached for speeding up some things.

So I've made Object Storage - a simple to use PHP library for storing objects as files.

Here are some of the ...

Highlights

Basic operations: set, get, delete, append, duplicate, rename and search.
Objects can have metadata. You can add multiple name-value pairs.
Atomic operations. A write operation ("set" for example) locks the object until it's done and other write operations cannot interfere with the one currently running.
Transactions. You can execute multiple actions on multiple objects that will be applied atomically. This is achieved by locking all files that need modification, prior execution.

Examples

$storage = new ObjectStorage('data/');

When you construct the Object Storage client, you provide the directory where the object will be saved. In that directory will be created three more directories:

objects/ - stores the objects as files
metadata/ - storage the objects metadata
temp/ - stores the library internal data

Setting data

$storage->set([
    'key' => 'users/1',
    'body' => '{"name":"John Smith","email":"john@example.com"}',
    'metadata.lastAccessTime' => (string) time()
]);

Getting data

$result = $storage->get([
    'key' => 'users/1',
    'result' => ['body', 'metadata']
]);
/*
If the object exists $result will be:
Array
(
    [body] => {"name":"John Smith","email":"john@example.com"}
    [metadata.lastAccessTime] => 1234567890
)
If the object does not exists $result will empty array.
*/

Appending data

$storage->append([
    'key' => 'visits/ip.log',
    'body' => "123.123.123.123\n"
]);

Duplicating data

$storage->duplicate([
    'sourceKey' => 'users/1',
    'targetKey' => 'users/2'
]);

Renaming data

$storage->rename([
    'sourceKey' => 'users/2',
    'targetKey' => 'users/3'
]);

Deleting data

$storage->delete([
    'key' => 'users/3'
]);

Searching for data

$result = $storage->search([
    'where' => [
        ['key', ['users/1']]
    ],
    'result' => ['key', 'body']
]);
/*
The result is array of arrays containing the matching objects
Array
(
    [0] => Array
        (
            [key] => users/1
            [body] => {"name":"John Smith","email":"john@example.com"}
        )
)
*/

// You can also use regular expresions
$result = $storage->search([
    'where' => [
        ['key', '^users\/', 'regexp']
    ],
    'result' => ['key', 'body']
]);

Multiple operations

$result = $storage->execute(
    [
        [
            'command' => 'set',
            'key' => 'users/9',
            'body' => '{"name":"John Smith","email":"john@example.com"}'
        ],
        [
            'command' => 'append',
            'key' => 'emails.log',
            'body' => 'john@example.com'
        ],
        [
            'command' => 'set',
            'key' => 'emails/' . md5('john@example.com'),
            'body' => '9'
        ],
        [
            'command' => 'get',
            'key' => 'users/9',
            'result' => ['body']
        ],
    ]
);
/*
The $result variable will be:
Array
(
    [0] => true // the result of the first operation
    [1] => true  // the result of the second operation
    [2] => true  // the result of the third operation
    [3] => Array  // the result of the fourth operation
        (
            [body] => {"name":"John Smith","email":"john@example.com"}
        )
)
*/

The source and the latest release are available at GitHub.

Speed up your website with CloudFront

Thu, 12 Nov 2015 11:19:55 +0000

We all love fast websites, and there are many techniques to make a website fast. In this post, you'll learn how to make your website faster by utilizing CloudFront. This website does. CloudFront is a CDN (Content Delivery Network) provided by Amazon Web Services.

You may have heard that CDNs are great for static content. They store it closer to the visitors (on edge servers), deliver it very fast and offload our server. It's true. And what about dynamic content? The dynamic content is actually a static content with TTL of 0. In this case, CloudFront will proxy the content and will not cache it.

What we are going to do now is position CloudFront in front of our website and make it cache content (or just proxy it) based on the headers (Cache-Control) sent from our server. It sounds easy and really is.

In the following example, we will speed up ivopetkov.com. The first thing is to make our website accessible via another domain. I'm using the subdomain origin.ivopetkov.com, which points to the exact same place as ivopetkov.com. Later we'll point ivopetkov.com to CloudFront, and point CloudFront to origin.ivopetkov.com.

Now it the best time to create AWS account if you don't have one.

Create a CloudFront distribution

Using the AWS Management Console find the CloudFront service and click Create Distribution.

In the first screen select "Web distribution".

The following settings makes the distribution proxy or cache everything based on the response headers from the server. It's a very basic configuration, but it gets the job done.

It may take up to 15 minutes for your distribution to be ready for use.

Using a CloudFront distribution

After your distribution is ready, you'll get a unique domain name (looks like abcdefghijk.cloudfront.net). You can open it, and you should see your website.

The next step is to point your visitors this way. Route 53 (the AWS DNS Service) makes this very easy. You just need to create an A type DNS record and make it an alias.

Cache headers

All you must do now is make sure your server returns correct Cache-Control header for every request. It usually looks like this:

Cache-Control: max-age=60, public

The results and the cost

The results depend on many things. Some of them are:

- how many requests are made on each pageview

- what amount of data is transferred on request

- where you visitors are located (how close to CloudFront edge location)

- how popular is your content (does it hit the CacheFront cache)

Some other performance benefits are:

- Persistent connections are used between your visitors and the edge servers and between the edge servers and your server. This makes loading more than one resource faster.

- You server take it easy when traffic spikes occur and you can actually lower your server bill.

- The cache-control headers tell CloudFront to cache the content, but users' browsers also cache it.

AWS offers a free tier so you can test this technique for free very easily. Take a look at the pricing too. This calculator will make estimating costs easier.

HTML Server Components

Thu, 10 Sep 2015 21:22:24 +0000

HTML was created a long time ago. We started building documents that contained only text, images and links. We enhanced them with CSS and JavaScript. We built image galleries with fancy animations, drop downs and many other UI elements. We started using Ajax. Now we use tools like jQuery, React, AngularJS in the browser. On the server, we've got frameworks, compilers, preprocessors. All these tools help us prepare the HTML, CSS and JavaScript code that browsers render to our visitors.

When we learn a new language, we start with a simple "Hello world". This is how it looks in HTML.


    
        Hi
    
    
        Hello world

But the real websites look like this

Hi

As you can see:

there are a lot of JS and CSS files included
there is a lot of inlined JS and CSS code
there is a lot of HTML code for different UI elements

The final code is messy, so we need tools to help us do our job.

Something new

Today I would like to present you a new concept for building HTML documents that are full of different components (fancy galleries, social share buttons, 3rd party comment boxes and so on). It looks a lot like React, WebComponents and Template engines, but it's also a little different.

Let's start with the following HTML code:


    
        Hi
    
    
        Hello world

When we add an image gallery with carousel effect we got the following code:

Hi

As you can see we as developers must do the following three things:

1. Include the JavaScript and CSS files of the library we've selected in the document head tag.

2. Write the HTML code in the body tag.

3. Call a JavaScript function to make the carousel work.

Often we write the different parts in different files, especially when we use frameworks and template engines. And this is where the things get really messy:

we include libraries more than once
we include things we don't use on every page
CSS and JavaScript conflicts occur

A simple concept

The concept is ... components, just like React and WebComponents, but on the server.

A component is just an HTML document (with and tags). We create one for each part of the HTML page (header, footer, carousel, product list, social plugin). Then we merge them into the final HTML result.

In this example, we've created two component files:

Hi



    
        
    
    
        Hello world

When we process them, we get


    
        Hi
        
    
    
        Hello world

Here is what happened

1. The tag is replaced with the content of the body tag from body.html

2. The style tag from body.html is moved to the head of the result document.

Simple, yet very powerful for real-world websites.

And of course, components can communicate using attributes.

And they can be nested too.

The benefits

All the code needed for one component to work is in the component's file. It's easy to develop, test and debug.
If the component is not used on the current page it's dependencies are not loaded (jQuery for example)
Components can be easily shared
Components can be easily reused and extended

It's easier, faster and simpler to develop websites using HTML Server Components. Need proof? Here is a sample web page build in PHP. View the source at GitHub. A very lightweight compiler (written in PHP) is used. It's available at GitHub too. It's only job is to merge the components by putting the different elements in the right places.

What's next

In this article, I presented you a concept that I call HTML Server Components, an example web page and a PHP compiler.

The things I've planned for future updates are:

1. CSS preprocessors (Less, Sass) integration

2. External JS and CSS files merging to minimize requests

3. Format options. You may want your output HTML pretty or minified

4. CSS and JavaScript minification

5. Inlining CSS and JavaScript files so there are fewer requests

6. Moving inlined CSS and JavaScript code to files

7. Template engines integration

Contribute

I'd love to hear your comments and ideas. The PHP compiler is really small and it's free on GitHub. If you want you can fork it or port it to other languages.

Lazy load responsive images

Wed, 19 Aug 2015 19:27:51 +0000

Note: This article is still useful but there is a new version with an updated API.

Let's start with a brief intro if you've never heard of lazy loading HTML images.

Lazy loading is a technique that defers image loading until it becomes visible on the screen. This saves bandwidth and can drastically improve page loading time and user experience.

There are few ways to implement lazy image loading. The most popular of them looks like this:

The blank.gif is a 1x1 transparent image file that loads very quickly because of it's tiny size. Then JavaScript code checks if the image is visible on the screen (after page load or page scroll) and replaces the value of the src attribute with the value of the data-src attribute. This triggers the loading of the original image. It's a very simple and powerful implementation, but it has some drawbacks.

One of the them is the lack of responsive images support. Hopefully, this can be easily fixed. The updated version looks like this:

The available image versions with their corresponding width are listed in the data-srcset attribute, and the appropriate image is selected.

Another big drawback is the blank.gif file itself. This image is bad for SEO, social networks sharing and read-it-later tools. You'll miss valuable traffic from Google Images, your posts in Facebook will not contain images (unless og:image meta tag is specified), they will not be shareable on Pinterest and the important graphics in your article will not be visible in Pocket. And again there is a fix:

The small-image.jpg file is a scaled down version of the original image. Let's say 200px in width and 10kb in size.

Loading this image is faster than loading the original image but still requires bandwidth and another HTTP request, so it's not the perfect solution.

Also, it will be great to set the image size to avoid reflowing when the original image url is set. Here is an updated version for fixed-sized images:

and a version for fluid (responsive) ones:

Read about this technique here.

And now it's time for ...

The best implementation

The following implementation:

- is called Responsively Lazy (available on GitHub)

- handles responsive images

- does not make ANY unnecessary requests

- works on modern browsers that support srcset. As of December 2017 that's 86.78%.

- handles loaded image resizes (when the phone is rotated from portrait to landscape). It's a nice bonus.

- does not break the page when an old browser is used

- works great in read-it-later tools like Pocket

- works great in content sharing on social networks like Facebook and Pinterest

- supports WebP

Here is an example:

And this is how the magic happens:

- The responsively-lazy class and the padding-bottom style save space for the image and eliminate reflowing. The padding value is calculated using the following formula: imageHeight/imageWidth*100. Learn more here. A different HTML code is available at the end of the article in case you don't know the image size.

- The data:image in the srcset attribute shows a transparent image and prevents loading the image in the src attribute in modern browsers. That's the most important part.

- The image in the src is attribute is used if the browser does not support the srcset attribute or when the page is scanned by social networks and read-it-later tools.

- The data-srcset attribute contains the available image versions with their corresponding width. The best version is selected based on the space available and the device pixel ratio.

A brand new concept

Other lazy loading libraries make you break your HTML by removing the src attribute, or make you put tiny version there or make you use

Responsively Lazy enhances it without breaking it.

Customize

You can drop the div tag and move the responsively-lazy class name to the img tag if you don't know the image aspect ratio.

VIEW DEMO

The source is available at https://github.com/ivopetkov/responsively-lazy/

So, are you ready to use lazy loading for your website's images?

Installing SSL certificate on Apache

Thu, 30 Jul 2015 19:56:18 +0000

Now you know how SSL works and how you can get one. It's time to secure our website connection.

The first thing we need to do is ensure the private key, the certificate, and the CA's intermediate certificates are located on the machine where the website is hosted but not in the website root directory. We must keep the private key secure.

Here is how the private key (located at /var/certificates/example.com.key) file should look like:

-----BEGIN PRIVATE KEY-----
MIIEvgIBADANx715AmXSr8RTIQfIgAAECggEANYzLwNPLOlRfwE81BEoeBgkqklW
...
UshV/35RY677Xe739BYnl+hZO5DBGNPeU/grdKdTqyZD/b2DtS5bK
-----END PRIVATE KEY-----

This is how the certificate file (located at /var/certificates/example.com.crt) should look like:

-----BEGIN CERTIFICATE-----
MIIFQjCCBCa9qjIx2VwiQeoLBzE0pahyM4Z9MJLNE5tOuQOIFZhbGlkYXRlZg7EB
...
HLmJnggdhbqMjQjZWipQ
-----END CERTIFICATE-----

And finally this is how the intermediates file (located at /var/certificates/example.com.chain) should look like:

-----BEGIN CERTIFICATE-----
MIIFdDCCBFygAwIB01PRE8gUlNBIENlcngKCAgEAkehUktIKVrGsDKCAgEAkehUk
...
pQ8pTIqXOi6YEbvFScL
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIGCDCCA/CgAw1jSz9AdDTScBkxwtiBhcNMjkwMjExMjM1OTU5WjASIwDQYJKoZ
...
hZ7Q7drNJ3gjIVq4rXmsX
-----END CERTIFICATE-----

As you can see, we've concatenated the intermediate certificates provided by our Certificate Authority.

The next step is to edit the Apache configuration file (httpd.conf). Find or create that is configured to accept secure connections on port 443 and add the highlighted lines:


   ServerName example.com
   DocumentRoot "/var/www"
   SSLEngine on
   SSLCertificateFile /var/certificates/example.com.crt
   SSLCertificateKeyFile /var/certificates/example.com.key
   SSLCertificateChainFile /var/certificates/example.com.chain
   SSLProtocol All -SSLv2 -SSLv3
   SSLHonorCipherOrder on
   SSLCipherSuite "EECDH+ECDSA+AESGCM EECDH+aRSA+AESGCM EECDH+ECDSA+SHA384 EECDH+ECDSA+SHA256 EECDH+aRSA+SHA384 EECDH+aRSA+SHA256 EECDH+aRSA+RC4 EECDH EDH+aRSA !RC4 !aNULL !eNULL !LOW !3DES !MD5 !EXP !PSK !SRP !DSS"

Then test the configuration file using the following command

apachectl configtest

and if the file is valid run the following command to apply the new configuration

apachectl graceful

That's it. Now visitors can access your website securely.

Getting SSL certificate

Sun, 26 Jul 2015 07:46:07 +0000

In the previous post How SSL works, we learned what the private key and the certificate are and how they can secure our website connection. Now it's time to get a pair. These are the steps needed:

1. Generate a private key

2. Generate a Certificate Signing Request (CSR)

3. Get a certificate

Currently, the easiest and most secure method to generate a private key and CSR is on a Linux machine. All you need is the openssl library. Run this command to ensure it's installed.

yum install openssl

Generate private key

The following command will generate a 2048 bit long private key and write it to a file named example.com.key.

openssl genrsa -out example.com.key 2048

Generate Certificate Signing Request (CSR)

The following command will create a file named example.com.csr containing your Certificate Signing Request using the key created in the previous step.

openssl req -new -sha256 -key example.com.key -out example.com.csr

During the file creation, you will be requested to enter some information. Here is an example:

You are about to be asked to enter information that will be incorporated into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:US
State or Province Name (full name) []:New York
Locality Name (eg, city) [Default City]:New York City
Organization Name (eg, company) [Default Company Ltd]:My Company Ltd
Organizational Unit Name (eg, section) []:IT
Common Name (eg, your name or your server's hostname) []:www.example.com
Email Address []:support@example.com

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:

The Common Name field is the place where you must enter the domain name you want to secure. The typical values are www.example.com and *.example.com.

When requested for a password just press enter to leave the field blank.

When created, this file will contain the public key and the information you've entered.

Get the certificate

If you plan to use the certificate only for developer purposes you can sign it yourself for free. This types of certificates trigger warnings in the browsers, so they are not a valid solution for public websites. The following command will generate a self-signed certificate in a file named example.com.crt that will be valid for 365 days.

openssl req -x509 -days 365 -in example.com.csr -key example.com.key -out example.com.crt

If you need to secure a public website, you must find a Certificate Authority to issue (sign with their private key) a certificate that can be validated by the browsers. This is a paid service, and here are some popular authorities: NameCheap, Comodo, DigiCert. They all offer different validation options. The most common and cheap one is the domain validation option. With this type of validation, you usually get a certificate within minutes, and all you must do is verify domain ownership.

Usually, these are the steps for domain validation only certificates:

1. You choose the certificate type (single domain, unlimited subdomains, etc.) and make a payment.

2. You fill a small form with the domain name and the content of your CSR file.

3. They send you a confirmation email at one of the following email addresses: admin@example.com, webmaster@example.com or other listed in the domain's WHOIS data.

4. You open the email and click a confirmation link.

5. Soon you receive another email containing your certificate and some other CA certificates called intermediate.

In the next post, we'll learn how to use the private key, the certificate, and the intermediate certificates to secure a website running on Apache.

How SSL works

Tue, 14 Jul 2015 09:05:29 +0000

SSL stands for Secure Sockets Layer. It's also known as Transport Layer Security (TLS). This is the technology used to provide security on the web. Today we will learn how SSL makes the communication between a web browser and a website secure so no one can see our passwords or other personal data while they are traveling through the Internet.

Terms we need to know before we start

Key

A combination of characters (like a password) but preferably long in size, so it is very very hard to guess.

Encryption

The process of encoding information so that it can only be decoded by using a special key.

Decryption

The process of decoding information by using a special key.

Private key

A key that can encrypt or decrypt information. As the name implies, it must be kept in a secret place.

Public key

A key that can encrypt or decrypt information but can be shared.

Certificate

File that contains a public key and other information.

Certificate authority

Organization that issues certificates and can help your browser verify them.

Certificate authority root certificates

Certificates that are securely stored on your device and help you verify the identity of authority issued certificates. Your OS vendor takes care of updating them.

The secure connection requires two keys to be created, one public, and one private. Through the magic of mathematics, they can be used to encrypt or decrypt information. Everything encrypted with the private key can only be decrypted with its corresponding public key and vice versa.

How secure communication is established

These are our main players:

- A browser that has access to the root certificates

- A web server that hosts the website. It stores a private key securely and has a valid certificate issued by a Certificate Authority.

The browser requests secure connection from the web server

The web server responds with its certificate

The browser verifies the certificate using the root certificates, so it is sure it's communicating with the right web server

Now both parties have valid keys they can use for secure communication. The browser is using the public key for decrypting server responses and sending encrypted requests. The server is using its private key for decrypting requests and encrypting responses.

That's it.

Please use the comment box bellow if you like this post or need more information.