Install private plugins and themes from Grav's admin interface

An short explanation and a debugging session.

This website is made with Grav, a CMS based on markdown files for content and modern PHP for rendering and the rest. I'm not doing anything crazy here, it's just a personal blog, but so far I'm happy with it. I created my own theme, named microblog, that covers my needs.

Today I wanted to fix a few styling and usability issues I have with my theme, so I started my local environment (just a Dockerfile to run an Nginx + php fpm to replicate the real environment), and did my changes. Once done, I faced the question: how do I distribute it to my server?

You have the following solutions:

  • copy through ssh, e.g: scp, or SFTP if configured (of course only possible if you have an ssh access). That's fast and simple, but be sure to use the correct user and/or set correct file permissions.
  • use GPM, the plugin system used by Grav. If your project is open source (specifically MIT licensed) then you can add it to the public list of plugins by opening an issue in GitHub, then install and keep it updated through Grav's admin panel.
  • distribute it as a ZIP'ed file either via the "Direct Install" option from the admin panel or via bin/gpm direct-install. Both are equivalent but the later requires CLI access.

I decided for the "Direct Install" approach, I find that works quite well in my use case. I can just do my change locally, bump the theme version in blueprints.yaml, zip then upload, and I'm done. And the zip + upload can be automated if too annoying. I don't want to manipulate my website content via SSH even though I have some Ansible to automate the website setup, and I don't see a reason to have my theme public and open source as I'm not willing to accept external contributions, though I really like the plugin system and would like to see the Grav team implement support for private registries.

Direct Install via the admin panel

It took me a bit of time and reverse engineering of Grav sources to understand what I'm supposed to zip and upload, I couldn't find anything on this online.

Based on what I found it, it should be done like this:

  1. Create your theme locally
  2. Don't forget to set a blueprints.yaml with correct information
  3. From the user/themes directory, remove anything that isn't your own theme directory. In my case I had to delete a .gitkeep file.
  4. Then zip the user/themes directory directly, not just your own theme folder! I continuously got an error Not a valid Grav package because I wasn't zipping the correct directory.
  5. Upload the resulting themes.zip to your website, and that should be good.

Debugging the "Not a valid Grav package" error

Note: I have almost zero experience with PHP, but the flow is fairly straightforward to follow.

After looking at it I feel that the direct install isn't as robust as it could be, the code responsible to determine if an upload is valid is here, the error message when running the direct install from the CLI made it very easy to isolate.

A short analysis allows us to make the following observations.


$this->output->write('  |- Extracting package...    ');
$extracted = Installer::unZip($zip, $tmp_source);

Quite simple, here the uploaded archive is being unzipped, though without reading its implementation it's not clear to me what that function returns. Before digging deeper, I added a few print statements to understand what $extracted is set to after the call to Installer::unZip, it seems to be the name of the first directory (or first file of first directory?) contained in the archive. For example when I uploaded user/themes/microblog.zip, $extracted was set to ... tmp/Grav-5f199d1b031e7/assets/site.webmanifest (???). Seems weird as I would expect the result of something named unZip to be either a directory object, an archive object, or the path to the root of the unzipped files, but let's continue the flow as that's not where the error was printed.


$type = GPM::getPackageType($extracted);

Ok so that statement is the one that when asserted to false, results in the error message we are debugging. Again, I just printed $type to see what it is, and it seems to be false with the failing upload.

So let's digg into those function implementations.

    /**
    * Unzip a file to somewhere
     *
     * @param string $zip_file
     * @param string $destination
     * @return bool|string
     */
    public static function unZip($zip_file, $destination)
    {
        $zip = new \ZipArchive();
        $archive = $zip->open($zip_file);

        if ($archive === true) {
            Folder::create($destination);

            $unzip = $zip->extractTo($destination);


            if (!$unzip) {
                self::$error = self::ZIP_EXTRACT_ERROR;
                Folder::delete($destination);
                $zip->close();
                return false;
            }

            $package_folder_name = preg_replace('#\./$#', '', $zip->getNameIndex(0));
            $zip->close();

            return $destination . '/' . $package_folder_name;
        }

        self::$error = self::ZIP_EXTRACT_ERROR;
        self::$error_zip = $archive;
        return false;
    }

In general that does what I would expect, create the temp directory, extract archive content, return resulting directory. From my point of view, the only suspicious part is preg_replace('#\./$#', '', $zip->getNameIndex(0)). The fact that we take a first index seems weird, I guess the idea is to only try to import the first plugin (or theme) directory found. In combination with the regexp that would explain why my upload is failing, the first index from the archive seems to be assets/site.webmanifest.

I find the logic here a bit surprising, I would expect the following:

  • unarchive, traverse the extracted content to find directory containing blueprints.yaml
  • install each plugin found this way

No need for a regexp or hard coded index, which offers a better user experience (possibility to import multiple plugins) and is less error prone. If only one plugin per import should be supported, either ask the user which one they want (in both the CLI and the web interface), or stop at the first one.

Now the next function, getPackageType.


     /**
     * Try to guess the package type from the source files
     *
     * @param string $source
     * @return bool|string
     */
    public static function getPackageType($source)
    {
        $plugin_regex = '/^class\\s{1,}[a-zA-Z0-9]{1,}\\s{1,}extends.+Plugin/m';
        $theme_regex = '/^class\\s{1,}[a-zA-Z0-9]{1,}\\s{1,}extends.+Theme/m';

        if (
            file_exists($source . 'system/defines.php') &&
            file_exists($source . 'system/config/system.yaml')
        ) {
            return 'grav';
        }

        // must have a blueprint
        if (!file_exists($source . 'blueprints.yaml')) {
            return false;
        }

        // either theme or plugin
        $name = basename($source);
        if (Utils::contains($name, 'theme')) {
            return 'theme';
        }
        if (Utils::contains($name, 'plugin')) {
            return 'plugin';
        }
        foreach (glob($source . '*.php') as $filename) {
            $contents = file_get_contents($filename);
            if (preg_match($theme_regex, $contents)) {
                return 'theme';
            }
            if (preg_match($plugin_regex, $contents)) {
                return 'plugin';
            }
        }

        // Assume it's a theme
        return 'theme';
    }

The function returns either grav, theme, plugin, or false. I guess PHP doesn't have enums? Anyway, a few notes.

That's where we do the check for the blueprints.yaml with if (!file_exists($source . 'blueprints.yaml')) { /* ... */ }. That's fine in itself though I wouldn't expect the validation to be done by something named as "tell me what type of package that is". With the approach I suggested before, traversing to look for blueprints, that could be removed and this function would become just a simple check between grav, theme and plugins.

// Assume it's a theme, that's just ... weird? Why not return false by default? Looks like a strange choice to make, seems quite error prone.

So that's it for this analysis. With the information we got we now know see what directory should be zipped, it should be either plugins or themes, and it should contain only one single plugin/theme directory, otherwise it has a risk to fail.

Conclusion

So writing this debugging session down took way more time than I initially expected, that's something I should keep in mind when I decide to write future posts. In itself the debugging was a short process. I don't know if I want to spend more time on this topic but I may follow-up with a pull request to improve a bit Grav's user experience, or at least open an issue on their GitHub repository. If I do I will add a link here. In any case I'm happy that I found out the correct way to update my theme without too much hassle. Grav codebase is quite nice, I appreciate, I know nothing about the language and I can follow nicely what is happening, it was interesting to peek a bit into their work.

The plugin management from the admin panel is awesome in my opinion, it is simple to use from both the web interface and command line tool. It's a bit of a shame that it doesn't support private registries. I hope that will come in the future.