Developing a PHP package, 2025 edition 16 Jan 11:27 PM (2 months ago)
Since my blog post about the anatomy of a dope PHP package repository a lot has changed in the PHP ecosystem and it's surrounding tools. Time to reflect these changes and provide, yet another, guide to develop a PHP package in 2025.
Idea to implementation
The hardest part to get started is finding an idea worth implementing. Sources of inspiration can be visiting conferences, visiting local user groups, or the good old blog post reading. Also, daily business might generate an idea worth investing time. The idea might be very niche at first but that shouldn't stop you from going for it; the learnings might advance your self-confidence and career.Getting started
Template repositories are a GitHub feature released in June 2019, which allows you and others to generate new Git repositories with the same directory structure and files based on the given template repository. Some templates to look into are spatie/package-skeleton-php and ergebnis/php-package-template. If you're developing a dedicated Laravel package spatie/package-skeleton-laravel is a good starting point. For checking if a package is PDS conform, there are for one the command-line tools of the pds/skeleton project or the package analyser which is a very opinionated validator/analyser.Must have development tools
Some of the above-mentioned package templates come with preconfigured development tools like PHPUnit or the PHP Coding Standards Fixer. Other tools you might have to add or switch yourself. For example when you're developing a Laravel package you might want to replace PHPUnit with the traction gaining Pest testing framework and the PHP Coding Standards Fixer with Pint. For aiding you in development, tools like PHPStan and Rector have become quite mandatory. Also, worth checking out is their thriving extension ecosystem e.g. Larastan and rector-laravel. For the CI part, GitHub Actions have replaced former CI environments like Travis CI. So it might be a good invest to look into the usage and definition of GitHub Actions. The automation of dependency updates for dependant Composer packages can currently be handled via Dependabot but might soon be replaced by Conductor. Identifying wording or grammar mistakes in repositories might become easier once we can add Peck, a wrapper around GNU Aspell, into the mix. To practice Datensparsamkeit you can add the lean-package-validator, which ensures that no unnecessary package artifacts end up in the distributed dist archives. For some additional development tools, it's also worth checking out Tomas Votruba's tools selection.Filling the gaps with polyfills
In case you're stuck with an older PHP or PHPUnit version, there are polyfills (polyfill-php83, polyfill-php84, and PHPUnit-Polyfills) available for filling the gaps; which eases later migrations to current PHP or PHPUnit versions.Registering your package
After having finished the implementation of the package you need to Git tag it, ideally following semantic versioning, and submit it at Packagist, the main PHP package repository.Package promotion
To promote your package start with a care- and heartful crafted README.md file. If you want to rise the visibility of your package invest some time to create a catching logo or delegate its creation to your bubble. Other channels to promote your work are X, the awesome PHP list, and giving a presentation about it at your local user group.
Automating the backslash prefixing for native PHP function calls 23 Nov 2023 4:21 AM (last year)
After reading the blog post Why does a backslash prefix improve PHP function call performance by Jeroen Deviaene I was looking for a way to automate it for the codebase of the Lean Package Validator, to shave off some miliseconds for it's CLI. The PHP Coding Standards Fixer has a rule named native_function_invocation which does the very exact task.
Configuring the PHP Coding Standards Fixer
.php-cs-fixer.php
<?php
use PhpCsFixer\Config;
use PhpCsFixer\Finder;
$finder = Finder::create()
->in([__DIR__, __DIR__ . DIRECTORY_SEPARATOR . 'tests']);
$rules = [
'psr_autoloading' => false,
'@PSR2' => true,
'phpdoc_order' => true,
'ordered_imports' => true,
'native_function_invocation' => [
'include' => ['@internal'],
'exclude' => ['file_put_contents']
]
];
$cacheDir = \getenv('HOME') ? \getenv('HOME') : __DIR__;
$config = new Config();
return $config->setRules($rules)
->setFinder($finder)
->setCacheFile($cacheDir . '/.php-cs-fixer.cache');
To make this rule executeable I needed to add the --allow-risky=yes option to the PHP Coding Standards Fixer calls in the two dedicated Composer scripts shown next.
composer.json
"scripts": {
"lpv:test": "phpunit",
"lpv:test-with-coverage": "export XDEBUG_MODE=coverage && phpunit --coverage-html coverage-reports",
"lpv:cs-fix": "php-cs-fixer --allow-risky=yes fix . -vv || true",
"lpv:cs-lint": "php-cs-fixer fix --diff --stop-on-violation --verbose --dry-run --allow-risky=yes",
"lpv:configure-commit-template": "git config --add commit.template .gitmessage",
"lpv:application-version-guard": "php bin/application-version --verify-tag-match=bin",
"lpv:application-phar-version-guard": "php bin/application-version --verify-tag-match=phar",
"lpv:static-analyse": "phpstan analyse --configuration phpstan.neon.dist",
"lpv:validate-gitattributes": "bin/lean-package-validator validate"
},
After running the lpv:cs-fix Composer script the first time the tests of the system under test started failing due to file_put_contents being prefixed with a backslash when using phpmock\MockBuilder's setName method, so I had to exclude it as shown in the PHP Coding Standards Fixer configuration above.
Documenting Composer scripts 15 Jan 2018 5:31 AM (7 years ago)
For open source projects I'm involved with, I developed the habit to define, and document the steady growing amount of repository and build utilities via Composer scripts. Having Composer scripts available makes it trivial to define aliases or shortcuts for complex and hard to remember CLI calls. It also lowers the barrier for contributors to start using these tools while helping out with fixing bugs or providing new features. Finally they're also simplifying build scripts by stashing away complexity.
Defining Composer scripts
If you've already defined or worked with Composer scripts or even their npm equivalents you can skip this section, otherwise the next code snippet allows you to study how to define these. The here defined Composer scripts range from simple CLI commands with set options (e.g. the test-with-coverage script) to more complex build utility tools (i.e. the application-version-guard script) which are extracted into specific CLI commands to avoid cluttering up the composer.json or even the .travis.yml.
composer.json
{
"scripts": {
"test": "phpunit",
"test-with-coverage": "phpunit --coverage-html coverage-reports",
"cs-fix": "php-cs-fixer fix . -vv || true",
"cs-lint": "php-cs-fixer fix --diff --stop-on-violation --verbose --dry-run",
"configure-commit-template": "git config --add commit.template .gitmessage",
"application-version-guard": "php bin/application-version --verify-tag-match"
}
}
Describing Composer scripts
Since Composer 1.6.0 it's possible to set custom script descriptions via the scripts-descriptions element like shown next. It's to point out here that the name of a description has to match the name of a defined custom Composer script to be recognised at runtime. On another note it's to mention that the description should be worded in simple present to align with the other Composer command descriptions.
composer.json
{
"scripts-descriptions": {
"test": "Runs all tests.",
"test-with-coverage": "Runs all tests and measures code coverage.",
"cs-fix": "Fixes coding standard violations.",
"cs-lint": "Checks for coding standard violations.",
"configure-commit-template": "Configures a local commit message template.",
"application-version-guard": "Checks that the application version matches the given Git tag."
},
"scripts": {
"test": "phpunit",
"test-with-coverage": "phpunit --coverage-html coverage-reports",
"cs-fix": "php-cs-fixer fix . -vv || true",
"cs-lint": "php-cs-fixer fix --diff --stop-on-violation --verbose --dry-run",
"configure-commit-template": "git config --add commit.template .gitmessage",
"application-version-guard": "php bin/application-version --verify-tag-match"
}
}
Now when running $ composer via the terminal the descriptions of defined custom scripts will show up sorted in into the list of available commands, which makes it very hard to spot the Composer scripts of the package at hand. Luckily Composer scripts can also be namespaced.Namespacing Composer scripts
To namespace (i.e. some-namespace) the custom Composer scripts for any given package define the script names with a namespace prefix as shown next. As the chances are very high that you will be using the one or other Composer script several times, while working on the package, it's recommended to use a short namespace like in the range from two to four characters.
composer.json
{
"scripts": {
"some-namespace:test": "phpunit",
"some-namespace:test-with-coverage": "phpunit --coverage-html coverage-reports",
"some-namespace:cs-fix": "php-cs-fixer fix . -vv || true",
"some-namespace:cs-lint": "php-cs-fixer fix --diff --stop-on-violation --verbose --dry-run",
"some-namespace:configure-commit-template": "git config --add commit.template .gitmessage",
"some-namespace:application-version-guard": "php bin/application-version --verify-tag-match"
}
}
Now this time when running $ composer via the terminal the defined custom scripts will show up in the list of available commands in a namespaced manner giving an immediate overview of the available Composer script of the package at hand.$ composer ... ommitted content Available commands: ... ommitted content some-namespace some-namespace:application-version-guard Checks that the application version matches the given Git tag. some-namespace:configure-commit-template Configures a local commit message template. some-namespace:cs-fix Fixes coding standard violations. some-namespace:cs-lint Checks for coding standard violations. some-namespace:test Runs all tests. some-namespace:test-with-coverage Runs all tests and measures code coverage.To use any namespaced Composer script, e.g. to fix coding standard violations after a substantial refactoring, it has to be called with its namespace e.g.$ composer some-namespace:cs-fix, which is the one disadavantage of Composer script namespacing.
Keeping your CLI integration tests green on Windows 25 Mar 2017 10:50 AM (8 years ago)
Lately on a Windows system, some failing integration tests for CLI commands utilising the Symfony Console component caused me some blip headaches by PHPUnit insisting that two strings are not identical due to different line endings. The following post documents the small steps I took to overcome these headaches.
First the assertion message produced by the failing test, see the console output below, got me thinking it might be caused by different encodings and line endings; though the project was utilising an .editorconfig from the early start and the related files were all encoded correctly and had the configured line endings. The Git configuration e.g. core.autocrlf=input also was as it should be.
1) Stolt\LeanPackage\Tests\Commands\InitCommandTest::createsExpectedDefaultLpvFile Failed asserting that two strings are identical. --- Expected +++ Actual @@ @@ #Warning: Strings contain different line endings! -Created default 'C:\Users\stolt\AppData\Local\Temp\lpv\.lpv' file. +Created default 'C:\Users\stolt\AppData\Local\Temp\lpv\.lpv' file.Another deeper look at the CommandTester class yielded that it’s possible to disable the command output decoration and also to normalise the command output. So a change of the SUT preparation and a normalisation of the console output, visualised via a git diff -U10, brought the solution for this particular test.
diff --git a/tests/Commands/InitCommandTest.php b/tests/Commands/InitCommandTest.php
index 58e7114..fb406f3 100644
--- a/tests/Commands/InitCommandTest.php
+++ b/tests/Commands/InitCommandTest.php
@@ -48,21 +48,21 @@ class InitCommandTest extends TestCase
/**
* @test
*/
public function createsExpectedDefaultLpvFile()
{
$command = $this->application->find('init');
$commandTester = new CommandTester($command);
$commandTester->execute([
'command' => $command->getName(),
'directory' => WORKING_DIRECTORY,
- ]);
+ ], ['decorated' => false]);
// ommitted code
- $this->assertSame($expectedDisplay, $commandTester->getDisplay());
+ $this->assertSame($expectedDisplay, $commandTester->getDisplay(true));
$this->assertTrue($commandTester->getStatusCode() == 0);
$this->assertFileExists($expectedDefaultLpvFile);Since the SUT had a lot of integration test for its CLI commands, the lazy me took the shortcut to extend the CommandTester and using it, with desired defaults set, instead of changing all of the related command instantiations.<?php
namespace SUT\Tests;
use Symfony\Component\Console\Tester\CommandTester as ConsoleCommandTester;
class CommandTester extends ConsoleCommandTester
{
/**
* Gets the display returned by the last execution of the command.
*
* @param bool $normalize Whether to normalize end of lines to \n or not
*
* @return string The display
*/
public function getDisplay($normalize = true)
{
return parent::getDisplay($normalize);
}
/**
* Executes the command.
*
* Available execution options:
*
* * interactive: Sets the input interactive flag
* * decorated: Sets the output decorated flag
* * verbosity: Sets the output verbosity flag
*
* @param array $input An array of command arguments and options
* @param array $options An array of execution options
*
* @return int The command exit code
*/
public function execute(
array $input,
array $options = ['decorated' => false]
) {
return parent::execute($input, $options);
}
}So it's a yay for green CLI command integration tests on Windows from here on. Another measure for the SUT would be to enable Continuous Integration on a Windows system via AppVeyor,
Eight knobs to adjust and improve your Travis CI builds 10 Oct 2016 1:02 PM (8 years ago)
After having refactored several Travis CI configuration files over the last weeks, this post will provide eight adjustments or patterns immediately applicable for faster, changeable, and economic builds.
1. Reduce git clone depth
The first one is a simple configuration addition with a positive impact on the time and disk space consumption, which should be quite noticeable on larger code bases. Having this configured will enable shallow clones of the Git repository and reduce the clone depth from 50 to 2..travis.yml
git: depth: 2
2. Enable caching
The second one is also a simple configuration addition for caching the Composer dependencies of the system under build (SUB) or its result of static code analysis. Generally have a look if your used tools allow caching and if so cache away. This one deserves a shout out to @localheinz for teaching me about this one.The next shown configuration excerpt assumes that you lint coding standard compliance with the PHP Coding Standards Fixer in version 2.0.0-alpha and have enable caching in its .php_cs configuration.
.travis.yml
cache:
directories:
- $HOME/.composer/cache
- $HOME/.php-cs-fixer
3. Enforce contribution standards
This one might be a tad controversial, but after having had the joys of merging GitHub pull requests from a master branch I started to fail builds not coming from feature or topic branch with the next shown bash script. It's residing in an external bash script to avoid the risk of terminating the build process../bin/travis/fail-non-feature-topic-branch-pull-request
#!/bin/bash set -e if [[ $TRAVIS_PULL_REQUEST_BRANCH = master ]]; then echo "Please open pull request from a feature / topic branch."; exit 1; fi
.travis.yml
script: - ./bin/travis/fail-non-feature-topic-branch-pull-requestThe bash script could be extended to fail pull requests not following a branch naming scheme, e.g. feature- for feature additions or fix- for bug fixes, by evaluating the branch name. If this is a requirement for your builds you should also look into the blocklisting branches feature of Travis CI.
4. Configure PHP versions in an include
With configuring the PHP versions to build against in a matrix include it's much easier to inject enviroment variables and therewith configure the version specific build steps. You can even set multiple enviroment variables like done on the 7.0 version..travis.yml
env:
global:
- OPCODE_CACHE=apc
matrix:
include:
- php: hhvm
- php: nightly
- php: 7.1
- php: 7.0
env: DISABLE_XDEBUG=true LINT=true
- php: 5.6
env:
- DISABLE_XDEBUG=true
before_script:
- if [[ $DISABLE_XDEBUG = true ]]; then
phpenv config-rm xdebug.ini;
fi
I don't know if enviroment variable injection is also possible with the minimalistic way to define the PHP versions list, so you should take that adjustment with a grain of salt.It also seems like I stumbled upon a Travis CI bug where the global enviroment variable OPCODE_CACHE is lost, so add another grain of salt. To work around that possible bug the relevant configuration has to look like this, which sadly adds some duplication and might be unsuitable when dealing with a large amount of environment variables.
.travis.yml
matrix:
include:
- php: hhvm
env:
- OPCODE_CACHE=apc
- php: nightly
env:
- OPCODE_CACHE=apc
- php: 7.1
env:
- OPCODE_CACHE=apc
- php: 7.0
env: OPCODE_CACHE=apc DISABLE_XDEBUG=true LINT=true
- php: 5.6
env: OPCODE_CACHE=apc DISABLE_XDEBUG=true
before_script:
- if [[ $DISABLE_XDEBUG = true ]]; then
phpenv config-rm xdebug.ini;
fi
5. Only do static code analysis or code coverage measurement once
This one is for reducing the build duration and load by avoiding unnecessary build step repetition. It's achived by linting against coding standard violations or generating the code coverage for just a single PHP version per build, in most cases it will be the same for 5.6 or 7.0..travis.yml
matrix:
include:
- php: hhvm
- php: nightly
- php: 7.1
- php: 7.0
env: DISABLE_XDEBUG=true LINT=true
- php: 5.6
env:
- DISABLE_XDEBUG=true
script:
- if [[ $LINT=true ]]; then
composer cs-lint;
composer test-test-with-coverage;
fi
6. Only do release releated analysis and checks on tagged builds
This one is also for reducing the build duration and load by targeting the analysis and checks on tagged release builds. For example if you want to ensure that your CLI binary version, the one produced via the --version option, matches the Git repository version tag run this check only on tagged builds..travis.yml
script:
- if [[ ! -z "$TRAVIS_TAG" ]]; then
composer application-version-guard;
fi
7. Run integration tests on very xth build
For catching breaking changes in interfaces or API's beyond your control it makes sense do run integration tests against them once in a while, but not on every single build, like shown in the next Travis CI configuration excerpt which runs the integration tests on every 50th build..travis.yml
script:
- if [[ $(( $TRAVIS_BUILD_NUMBER % 50 )) = 0 ]]; then
composer test-all;
else
composer test;
fi
8. Utilise Composer scripts
The last one is all about improving the readability of the Travis CI configuration by extracting command configurations i.e. options into dedicated Composer scripts. This way the commands are also available during your development activitives and not hidden away in the .travis.yml file.composer.json
{
"__comment": "omitted other configuration",
"scripts": {
"test": "phpunit",
"test-with-coverage": "phpunit --coverage-html coverage-reports",
"cs-fix": "php-cs-fixer fix . -vv || true",
"cs-lint": "php-cs-fixer fix --diff --verbose --dry-run"
}
}
To ensure you don't end up with an invalid Travis CI configuration, which might be accidently committed, you can use composer-travis-lint a simple Composer script linting the .travis.yml with the help of the Travis CI API.
Happy refactoring.
Anatomy of a dope PHP package repository 20 Sep 2016 9:32 AM (8 years ago)
While contributing to Construct, maintained by Jonathan Torres, I gathered some insights and learnings on the characteristics of a dope PHP package repository. This post summarises and illustrates these, so that PHP package develeopers have a complementary guideline to improve existing or imminent package repositories. Jonathan Reinink did a good job in putting the PHP package checklist out there which provides an incomplete, but solid quality checklist for open-source PHP packages.
I'll distill the characteristics of a dope PHP package repository by looking at the repository artifacts Construct can generate for you when starting the development of a new PHP project or micro-package. The following tree command output shows most of the elements this post will touch upon. The artifacts in parenthese are optional and configurable from Construct but can nonetheless have an import impact on the overall package quality.
├── <package-name> │ ├── CHANGELOG.md │ ├── (CONDUCT.md) │ ├── composer.json │ ├── composer.lock │ ├── CONTRIBUTING.md │ ├── (.editorconfig) │ ├── (.env) │ ├── (.env.example) │ ├── (.git) │ │ └── ... │ ├── .gitattributes │ ├── (.github) │ │ ├── CONTRIBUTING.md │ │ ├── ISSUE_TEMPLATE.md │ │ └── PULL_REQUEST_TEMPLATE.md │ ├── .gitmessage │ ├── .gitignore │ ├── (.lgtm) │ ├── LICENSE.md │ ├── (MAINTAINERS) │ ├── (.php_cs) │ ├── (phpunit.xml.dist) │ ├── README.md │ ├── (docs) │ │ └── index.md │ ├── src │ │ └── Logger.php │ ├── tests │ │ └── LoggerTest.php │ ├── .travis.yml │ ├── (Vagrantfile) │ └── vendor │ └── ...
Definition of a dope PHP package repository
Before jumping into the details, let's define what could be considered as a dope package repository. Therefor, being lazy, I'm going to simply reword this classic quote from Michael Feathers> Clean code is code that is written by someone who cares.to
> A dope PHP package repository is one that is created and maintained by someone who cares.
Artifact categories
The next shown pyramid illustrates the three main categories the artifacts of a package repository will fall into.
Second there's the configuration for a myriad of tools like Git, GitHub, EditorConfig, Composer, the preferred testing framework, the preferred continuous inspection / integration platform such like Scrutinizer or Travis CI, and so forth.
The final category includes tools which ease the life of maintainers and potential contributors equally. These tools can be helpful for releasing new versions, enforcing coding standard compliance, or commit message quality and consistency.
Consistency
Sourcecode
All sourcecode and accompanying tests or specs should follow a coding standard (PSR-2) and have a consistent formatting style, there's nothing new here. The perfect place to communicate such requirements is the CONTRIBUTING.md file.Tools like PHP Coding Standards Fixer or PHP_CodeSniffer in combination with a present configuration .php_cs|ruleset.xml.dist and a command wrapping Composer script are an ideal match to ease compliance. The Composer script cs-fix shown next will be available for maintainers and contributors alike.
composer.json
{
"__comment": "omitted other configuration",
"scripts": {
"cs-fix": "php-cs-fixer fix . -vv || true"
}
}
Consistent formatting styles like line endings, indentation style, and file encoding can be configured via an EditorConfig configuration residing in .editorconfig which will be used when supported by the IDE or text editor of choice.
Artifact naming and casing
Like sourcecode formatting and naming, repository artifacts should also follow a predictable naming scheme. All documentation files should have a consistent extension like .md or .rst and the casing should be consistent throughout the package repository. Comparing├── <package-name> │ ├── changelog.md │ ├── code_of_conduct.md │ ├── ... │ ├── .github │ │ └── ... │ ├── LICENSE │ ├── Readme.md │ ├── roadmap.rstto
├── <package-name> │ ├── CHANGELOG.md │ ├── CODE_OF_CONDUCT.md │ ├── ... │ ├── .github │ │ └── ... │ ├── LICENSE.md │ ├── README.md │ ├── ROADMAP.mdI would favour the later one anytime for it's much easier reading flow and pattern matchableness. The easier reading flow is achieved by the upper casing of the *.md files which also clearly communicates their documentation character.
The configuration files for tools which except the .dist file extension per default should all have such one like shown next.
├── <package-name> │ ├── build.xml.dist │ ├── phpunit.xml.dist │ ├── ruleset.xml.dist │ ├── ...
Commit message format
Next to the package's changelog, incrementally growing in the CHANGELOG.md file, the Git commit messages are an important source of change communication. Therefor they should also follow a consistent format which improves the reading flow while also leaving a professional impression. This format can be documented once again in the CONTRIBUTING.md file or even better be provided via a .gitmessage file residing in the package's Git repository.Once more a Composer script, named configure-commit-template here, can ease configuration and if configured Git will use it's content when committing without the -m|--message and -F|--file option.
composer.json
{
"__comment": "omitted other configuration",
"scripts": {
"configure-commit-template": "git config --add commit.template .gitmessage"
}
}To enforce commit message formatting adherence to the rules described by Chris Beams on a Git hook level, the git-lint-validators utility by Billie Thompson can be helpful.
Versioning
Release versions should follow the semantic versioning specification aka SemVer, once again there's nothing new here. When using version numbers in the sourcecode or CLI binaries, these should be in sync with the set Git tag. Tools like RMT or self-written tools should be utilised for this mundane task.The next shown code illustrates such a simple self-written tool named application-version. It's main purpose is to set the provided version number in the CLI application's binary and avoid an application version and Git tag mismatch.
bin/application-version
#!/usr/bin/env php
<?php
$binApplicationName = '<bin-application-name>';
$binFile = __DIR__ . DIRECTORY_SEPARATOR . $binApplicationName;
list($void, $binFileRelative) = explode($binApplicationName, $binFile, 2);
$shortBinFilePath = $binApplicationName . $binFileRelative;
$options = getopt('v:ch', ['version:', 'current', 'verify-tag-match', 'help', 'current-raw']);
$help = <<<HELP
This command sets the version number in the {$shortBinFilePath} file:
Usage:
application-version [options]
Options:
-c, --current, --current-raw The current version number
--verify-tag-match Verify application version and Git tag match
-v, --version The version number to set
-h, --help Display this help message
HELP;
if (array_key_exists('h', $options) || array_key_exists('help', $options)) {
echo $help;
exit(0);
}
/**
* Return the application version.
*
* @param string $binFile File holding the application version.
* @return string
*/
function get_application_version($binFile) {
$matches = [];
$match = preg_match(
'/(\d+\.)?(\d+\.)?(\*|\d+)/',
file_get_contents($binFile),
$matches
);
return trim($matches[0]);
}
/**
* Return latest tagged version.
*
* @return string
*/
function get_latest_tagged_version() {
exec('git describe --tags --abbrev=0', $output);
return trim($output[0]);
}
if (array_key_exists('verify-tag-match', $options)) {
$applicationVersion = 'v' . get_application_version($binFile);
$latestGitTag = get_latest_tagged_version();
if ($applicationVersion === $latestGitTag) {
echo "The application version and Git tag match on {$latestGitTag}." . PHP_EOL;
exit(0);
}
echo "The application version {$applicationVersion} and Git tag {$latestGitTag} don't match." . PHP_EOL;
exit(1);
}
if (array_key_exists('current-raw', $options)) {
echo get_application_version($binFile) . PHP_EOL;
exit(0);
}
if (array_key_exists('c', $options) || array_key_exists('current', $options)) {
$applicationVersion = 'v' . get_application_version($binFile);
$latestGitTag = get_latest_tagged_version();
echo "Current version set in {$shortBinFilePath} is {$applicationVersion}." . PHP_EOL;
echo "Current tagged version {$latestGitTag}." . PHP_EOL;
exit(0);
}
if ($options === []) {
echo 'No options set.' . PHP_EOL;
exit(1);
}
$version = isset($options['version']) ? trim($options['version']) : trim($options['v']);
$fileContent = file_get_contents($binFile);
$fileContent = preg_replace(
'/(.*define.*VERSION.*)/',
"define('VERSION', '$version');",
$fileContent
);
file_put_contents($binFile, $fileContent);
echo "Set version in {$shortBinFilePath} to {$version}." . PHP_EOL;
exit(0);
The application-version tool could further be utilised in Travis CI builds, to avoid the earlier mentioned version differences, like shown in the next .travis.yml diggest. On an application version and Git tag mismatch the shown build script will break the build early..travis.yml
language: php # omitted other configuration script: # Verify application version and Git tag match - php bin/application-version --verify-tag-match # omitted other scripts
Lean builds
To speed up continuous integration builds, resource and time consuming extensions like Xdebug should be disabled when not required for measuring code coverage. The next shown before_script, tailored for Travis CI, is generated by Construct per default and might shave off a few build seconds and thereby provide a faster feedback..travis.yml
language: php # omitted other configuration before_script: - phpenv config-rm xdebug.ini || true # omitted other before_scriptsTo reduce email traffic, the email notifications send by Travis CI should be reduced to a minimum like shown next, or dependent on your workflow the could be disabled at all.
.travis.yml
language: php
# omitted other configuration
notifications:
email:
on_success: never
Something I really would love to be supported by Travis CI is a feature to ignore a set of definable artifacts which could be configured in a .buildignore file or the like. This way wording or spelling changes on non build relevant artifacts like the README.md wouldn't trigger a build and misspend resources and energy. There's a related GitHub issue and here's hope it will be revisited in the near future.
Lean releases
To keep releases (or dists in Composer lingo) of PHP projects or micro-packages as lean as possible, their repositories should contain a complete and valid .gitattributes file. With such a file present all export-ignored files will be excluded from release archives and thereby save a significant amount of bandwith and energy.The next code shows the content of such a .gitattributes file excluding non release relevant files like internal tools, configuration, and documentation artifacts. If for some reasons you require the complete source of a PHP project or micro-package you can bypass the default by using Composer's --prefer-source option.
.gitattributes
* text=auto eol=lf .editorconfig export-ignore .gitattributes export-ignore .github/ export-ignore .gitignore export-ignore .gitmessage export-ignore .php_cs export-ignore .travis.yml export-ignore bin/application-version export-ignore bin/release-version export-ignore bin/start-watchman export-ignore CHANGELOG.md export-ignore LICENSE.md export-ignore phpunit.xml.dist export-ignore README.md export-ignore tests/ export-ignoreTo validate the .gitattributes file of a PHP project or micro-package on the repository, Git HEAD, or build level the LeanPackageValidator CLI can be helpful.
Avoid badge posing
Badges, if used sparsely, are a handy tool for visualising some of the repository properties. It's definitely nice to immediately see the required PHP version, the current build status, or the latest version of the package as they save you manual look ups.Badges showing the amount of downloads, code coverage, or the chosen license are in my opinion kind of poserish, they cause unnecessary requests to the badge service, and are in case of the license even obsolete.
Why should you care about the dopeness of a PHP package repository?
Creating and maintaining a dope PHP package repository might have a positive impact on several levels. It can earn you some valuable Brownie points or even provide a conversation gambit when doing job interviews, simply because you showcase professionalism.Furthermore it's more likely to get valuable and high quality contributions from your second target audience when supportive documentation and tooling for things like issue creation, coding standards compliance, or Git commit message consistency are available.
It also might convince an end user, your main target audience, in using your package over a competitive one.
Le fini
So these were my recent insights and learnings on the anatomy of a dope PHP package repository. If you'r lucky to be attending this year's ZendCon, I recommend to catch Matthew Weier O'Phinney's session about Creating PHPantastic packages. I definitely be waiting for the related slides.Happy packaging.
Enforcing target descriptions within build files with a Git hook 22 Apr 2011 7:44 AM (13 years ago)
When automating mundane tasks of a project or development environment with a build tool like Phing or Ant, the driving build file will naturally accumulate several targets and tasks over time. To ease the build file acceptance within a team and at a later stage also the contribution rate by team members, it's crucial that all build targets have a description attribute to provide at least a rough outline of the build features at hand. When these attributes are in place the (potential) build file user will get such an outline by executing the build tool's list command (phing -l or ant -p). To get a better picture of the problem at hand imagine a project poorly covered with tests and your personal attitude towards extending it or just take a peek at the screenshot below showing a very poorly documented build file.
To overcome this accumulation of some sort of technical debt (i.e. poorly documented targets) there are various options at hand. The first one, not covered in this blog post, would be to add a pursuant test which verifies the existence of a description for every target/task of the build file under test. As it's very uncommon, at least from what I've heard, to have your build files covered by tests; the next thinkable approach would be to use a Git pre-commit hook to guard your repository/ies against the creeping in of such poorly documented build files.
The next listing shows such a Git hook (also available via GitHub) scribbled away in PHP, which detects any build file(s) following a common build file naming schema (i.e. build.xml|build.xml.dist|personal-build.xml|…) , prior to the actual commit. For every target element in the detected build file(s) it's then verified that it has a description attribute and that it's actual content is long enough to carry some meaning. If one of those two requirements aren't met, the commit is rejected while revealing the build file smells to the committer, so she can fix it, as shown in the outro screenshot. Happy build file sniffing.
#!/usr/bin/php
<?php
define('DEPENDENT_EXTENSION', 'SimpleXML');
if (!extension_loaded(DEPENDENT_EXTENSION)) {
$consoleMessage = sprintf(
"Skipping build file checks as the '%s' extension isn't available.",
DEPENDENT_EXTENSION
);
echo $consoleMessage . PHP_EOL;
exit(0);
}
define('MIN_TARGET_DESCRIPTION_LENGTH', 10);
define('TARGET_DESCRIPTION_ATTRIBUTE', 'description');
define('TARGET_NAME_ATTRIBUTE', 'name');
define('CHECK_DESCRIPTION_LENGTH', true);
$possibleBuildFileNames = array(
'build.xml.dist',
'build.xml-dist',
'build-dist.xml',
'build.xml',
'personal-build.xml'
);
$violations = getAllBuildFileViolationsOfCommit($possibleBuildFileNames);
fireBackPossibleViolationsAndExitAccordingly($violations);
function getAllBuildFileViolationsOfCommit(array $possibleBuildFileNames)
{
$filesOfCommit = array();
$gitCommand = 'git diff --cached --name-only';
exec($gitCommand, $filesOfCommit, $commandReturnCode);
$allViolations = array();
foreach ($filesOfCommit as $file) {
if (in_array(basename($file), $possibleBuildFileNames)) {
$violations = checkBuildFileForViolations($file);
if (count($violations) > 0) {
$allViolations[$file] = $violations;
}
}
}
return $allViolations;
}
/**
* @param array $allViolations
* @return void
*/
function fireBackPossibleViolationsAndExitAccordingly(array $allViolations)
{
if (count($allViolations) > 0) {
foreach ($allViolations as $buildFile => $violations) {
$buildFileConsoleMessageHeader = sprintf("Build file '%s':", $buildFile);
echo $buildFileConsoleMessageHeader . PHP_EOL;
foreach ($violations as $violationMessage) {
$buildFileConsoleMessageLine = sprintf(" + %s", $violationMessage);
echo $buildFileConsoleMessageLine . PHP_EOL;
}
}
if (count($allViolations) > 1) {
$rejectCommitConsoleMessage = sprintf(
"Therefore rejecting the commit of build files [ %s ].",
implode(', ', array_keys($allViolations))
);
} else {
$rejectCommitConsoleMessage = sprintf(
"Therefore rejecting the commit of build file [ %s ].",
implode(', ', array_keys($allViolations))
);
}
echo $rejectCommitConsoleMessage . PHP_EOL;
exit(1);
}
exit(0);
}
/**
* @param string $buildfile
* @return array
*/
function checkBuildFileForViolations($buildFile) {
if (!file_exists($buildFile)) {
return array();
}
$buildfileXml = file_get_contents($buildFile);
$buildXml = new SimpleXMLElement($buildfileXml);
$allBuildTargets = $buildXml->xpath("//target");
$violations = array();
if (count($allBuildTargets) > 0) {
$targetsWithNoDescription = $targetsWithTooShortDescription = array();
foreach ($allBuildTargets as $buildTarget) {
$actualTragetAttributes = $buildTarget->attributes();
$allUsedTragetAttributes = array();
$actualTargetName = null;
foreach ($actualTragetAttributes as $attribute => $value) {
$allUsedTragetAttributes[] = $attribute;
if ($attribute === TARGET_NAME_ATTRIBUTE) {
$actualTargetName = $value;
}
if (CHECK_DESCRIPTION_LENGTH === true &&
$attribute === TARGET_DESCRIPTION_ATTRIBUTE &&
strlen($value) < MIN_TARGET_DESCRIPTION_LENGTH)
{
$targetsWithTooShortDescription[] = $actualTargetName;
}
}
if (!in_array(TARGET_DESCRIPTION_ATTRIBUTE, $allUsedTragetAttributes)) {
$targetsWithNoDescription[] = $actualTargetName;
}
}
if (count($targetsWithNoDescription) > 0) {
if (count($targetsWithNoDescription) > 1) {
$violations[] = sprintf(
"Build targets [ %s ] don't have mandatory descriptions.",
implode(', ', $targetsWithNoDescription)
);
} else {
$violations[] = sprintf(
"Build target [ %s ] doesn't have a mandatory description.",
implode(', ', $targetsWithNoDescription)
);
}
}
if (count($targetsWithTooShortDescription) > 0) {
if (count($targetsWithTooShortDescription) > 1) {
$violations[] = sprintf(
"Build targets [ %s ] don't have an adequate target description length.",
implode(', ', $targetsWithTooShortDescription),
MIN_TARGET_DESCRIPTION_LENGTH
);
} else {
$violations[] = sprintf(
"Build target [ %s ] doesn't have an adequate target description length.",
implode(', ', $targetsWithTooShortDescription),
MIN_TARGET_DESCRIPTION_LENGTH
);
}
}
}
return $violations;
}
Measuring & displaying Phing build times with buildhawk 20 Nov 2010 6:30 AM (14 years ago)
Recently I installed a Ruby gem called buildhawk which allows to measure and display the build times of Rake driven builds. As I like the idea behind this tool a lot but mostly use Phing for build orchestration, it was time to explore the possibility to interconnect them both. In this blog post I'll show an implementation of an apposite Phing Logger gathering the buildhawk compatible build times via git note(s) and how to put the interplay between those two tools to work.
Logging on
As mentioned above the build time of each build is stored as a git note and associated to the repository's HEAD, reflecting the current state of the system under build (SUB), which assumes that the SUB is versioned via Git. The next shown Phing Logger (i.e. BuildhawkLogger) grabs the overall build time by hooking into the buildFinished method of the extended DefaultLogger class, transforms it into a buildhawk specific format and finally adds it as a git note.<?php
require_once 'phing/listener/DefaultLogger.php';
/**
* Writes a build event to the console and store the build time as a git notes in the
* project's repository HEAD.
*
* @author Raphael Stolt <raphael.stolt@gmail.com>
* @see BuildEvent
* @link https://github.com/xaviershay/buildhawk Buildhawk on GitHub
* @package phing.listener
*/
class BuildhawkLogger extends DefaultLogger {
/**
* @var string
*/
private $_gitNotesCommandResponse = null;
/**
* Behaves like the original DefaultLogger, plus adds the total build time
* as a git note to current repository HEAD.
*
* @param BuildEvent $event
* @see BuildEvent::getException()
* @see DefaultLogger::buildFinished
* @link http://www.kernel.org/pub/software/scm/git/docs/git-notes.html
*/
public function buildFinished(BuildEvent $event) {
parent::buildFinished($event);
if ($this->_isProjectGitDriven($event)) {
$error = $event->getException();
if ($error === null) {
$buildtimeForBuildhawk = $this->_formatBuildhawkTime(
Phing::currentTimeMillis() - $this->startTime
);
if (!$this->_addBuildTimeAsGitNote($buildtimeForBuildhawk)) {
$message = sprintf(
"Failed to add git note due to '%s'",
$this->_gitNotesCommandResponse
);
$this->printMessage($message, $this->err, Project::MSG_ERR);
}
}
}
}
/**
* Checks (rudimentary) if the project is Git driven
*
* @param BuildEvent $event
* @return boolean
*/
private function _isProjectGitDriven(BuildEvent $event)
{
$project = $event->getProject();
$projectRelativeGitDir = sprintf(
'%s/.git', $project->getBasedir()->getPath()
);
return file_exists($projectRelativeGitDir) && is_dir($projectRelativeGitDir);
}
/**
* Formats a time micro integer to buildhawk readable format.
*
* @param integer The time stamp
*/
private function _formatBuildhawkTime($micros) {
return sprintf("%0.3f", $micros);
}
/**
* Adds the build time as a git note to the current repository HEAD
*
* @param string $buildTime The build time of the build
* @return mixed True on sucess otherwise the command failure response
*/
private function _addBuildTimeAsGitNote($buildTime) {
$gitNotesCommand = sprintf(
"git notes --ref=buildtime add -f -m '%s' HEAD 2>&1",
$buildTime
);
$gitNotesCommandResponse = exec($gitNotesCommand, $output, $return);
if ($return !== 0) {
$this->_gitNotesCommandResponse = $gitNotesCommandResponse;
return false;
}
return true;
}
}Putting the Logger to work
As the buildhawk logger is available via GitHub you can easily grab it by issuing sudo curl -s http://gist.github.com/raw/707868/BuildhawkLogger.php -o $PHING_HOME/listener/BuildhawkLogger.php. The next step, making the build times loggable, is achieved by using the -logger command line argument of the Phing Cli and specifying the buildhawk logger name or the path to it. In case you want the buildhawk logger to be used per default (it behaves like the default logger if the SUB isn't Git driven/managed) you can also add it to the Phing shell script.The next console command issued in the directory of the SUB shows a Phing call utilizing the BuildhawkLogger, assumed it has been installed at $PHING_HOME/listener/BuildhawkLogger.php and not been made the default logger.
phing -logger phing.listener.BuildhawkLogger
Looking at them Phing build times
Now it's time to switch to buildhawk and let it finally perform it's designated task, rendering an with the commit SHAs, commit messages, and build times fed Erb template into an informative, viewable HTML page. To install it you simply have to run sudo gem install buildhawk and you're good to go.The next console command shows the buildhawk call issued in the SUB's directory to produce it's build time report page.
buildhawk --title 'Examplr' > examplr-build-times.htmlThe outro screenshot below gives you a peek at a rendered build time report.
Growling PHPUnit's test status 2 Jun 2010 7:05 PM (14 years ago)
Two years ago I blogged about a Xinc (R.I.P?) plugin that growls each build status for any via Xinc continuously integrated project. Since I'm using PHPUnit more and more lately, especially in continuous testing sessions (sprints without hitting the continuous integration server), my dependence on a fast and more visual feedback loop rose. In this post I'll provide an easy solution that meets these requirements by utilizing PHPUnit's test listener feature.
What's the motivation, yo?
While doing story or feature sprints embedded in a continuous testing approach I first used a combination of stakeout.rb and PHPUnit's --colors option to radiate the tests status, but soon wasn't that satisfied with the chosen route as it happened that the console window got superimposed with other opened windows (e.g. API Browser, TextMate etc.) especially on my 13,3" MacBook.To overcome this misery I decided to utilize PHPUnit's ability to write custom test listeners and to implement one that radiates the test status in a more prominent and sticky spot via Growl.
Implementing the Growl test listener
Similar to the ticket listener plugin mechanism I blogged about earlier PHPUnit also provides one for test listeners. This extension mechanism allows to bend the test result formatting and output to the given needs and scenarios a developer might face and therefore is a perfect match.To customize the test feedback and visualization the test listener has to implement the provided PHPUnit_Framework_Testlistener interface. A few keystrokes later I ended up with the next shown implementation, which is also available via a GitHub gist, supporting the previous stated requirements.
<?php
class PHPUnit_Extensions_TestListener_GrowlTestListener
implements PHPUnit_Framework_Testlistener
{
const TEST_RESULT_COLOR_RED = 'red';
const TEST_RESULT_COLOR_YELLOW = 'yellow';
const TEST_RESULT_COLOR_GREEN = 'green';
private $_errors = array();
private $_failures = array();
private $_incompletes = array();
private $_skips = array();
private $_tests = array();
private $_suites = array();
private $_endedSuites = 0;
private $_assertionCount = 0;
private $_startTime = 0;
private $_successPicturePath = null;
private $_incompletePicturePath = null;
private $_failurePicturePath = null;
/**
* @param string $successPicturePath
* @param string $incompletePicturePath
* @param string $failurePicturePath
*/
public function __construct($successPicturePath, $incompletePicturePath,
$failurePicturePath)
{
$this->_successPicturePath = $successPicturePath;
$this->_incompletePicturePath = $incompletePicturePath;
$this->_failurePicturePath = $failurePicturePath;
}
public function addError(PHPUnit_Framework_Test $test, Exception $e, $time)
{
$this->_errors[] = $test->getName();
}
public function addFailure(PHPUnit_Framework_Test $test,
PHPUnit_Framework_AssertionFailedError $e, $time)
{
$this->_failures[] = $test->getName();
}
public function addIncompleteTest(PHPUnit_Framework_Test $test,
Exception $e, $time)
{
$this->_incompletes[] = $test->getName();
}
public function addSkippedTest(PHPUnit_Framework_Test $test,
Exception $e, $time)
{
$this->_skips[] = $test->getName();
}
public function startTest(PHPUnit_Framework_Test $test)
{
}
public function endTest(PHPUnit_Framework_Test $test, $time)
{
$this->_tests[] = array('name' => $test->getName(),
'assertions' => $test->getNumAssertions()
);
$this->_assertionCount+= $test->getNumAssertions();
}
public function startTestSuite(PHPUnit_Framework_TestSuite $suite)
{
if (count($this->_suites) === 0) {
PHP_Timer::start();
}
$this->_suites[] = $suite->getName();
}
public function endTestSuite(PHPUnit_Framework_TestSuite $suite)
{
$this->_endedSuites++;
if (count($this->_suites) <= $this->_endedSuites)
{
$testTime = PHP_Timer::secondsToTimeString(
PHP_Timer::stop());
if ($this->_isGreenTestResult()) {
$resultColor = self::TEST_RESULT_COLOR_GREEN;
}
if ($this->_isRedTestResult()) {
$resultColor = self::TEST_RESULT_COLOR_RED;
}
if ($this->_isYellowTestResult()) {
$resultColor = self::TEST_RESULT_COLOR_YELLOW;
}
$suiteCount = count($this->_suites);
$testCount = count($this->_tests);
$failureCount = count($this->_failures);
$errorCount = count($this->_errors);
$incompleteCount = count($this->_incompletes);
$skipCount = count($this->_skips);
$resultMessage = '';
if ($suiteCount > 1) {
$resultMessage.= "Suites: {$suiteCount}, ";
}
$resultMessage.= "Tests: {$testCount}, ";
$resultMessage.= "Assertions: {$this->_assertionCount}";
if ($failureCount > 0) {
$resultMessage.= ", Failures: {$failureCount}";
}
if ($errorCount > 0) {
$resultMessage.= ", Errors: {$errorCount}";
}
if ($incompleteCount > 0) {
$resultMessage.= ", Incompletes: {$incompleteCount}";
}
if ($skipCount > 0) {
$resultMessage.= ", Skips: {$skipCount}";
}
$resultMessage.= " in {$testTime}.";
$this->_growlnotify($resultColor, $resultMessage);
}
}
/**
* @param string $resultColor
* @param string $message
* @param string $sender The name of the application that sends the notification
* @throws RuntimeException When growlnotify is not available
*/
private function _growlnotify($resultColor, $message = null, $sender = 'PHPUnit')
{
if ($this->_isGrowlnotifyAvailable() === false) {
throw new RuntimeException('The growlnotify tool is not available');
}
$notificationImage = $this->_getNotificationImageByResultColor(
$resultColor);
$command = "growlnotify -w -s -m '{$message}' "
. "-n '{$sender}' "
. "-p 2 --image {$notificationImage}";
exec($command, $response, $return);
}
/**
* @return boolean
*/
private function _isGrowlnotifyAvailable()
{
exec('growlnotify -v', $reponse, $status);
return ($status === 0);
}
/**
* @param string $color
* @return string
*/
private function _getNotificationImageByResultColor($color)
{
switch ($color) {
case self::TEST_RESULT_COLOR_RED:
return $this->_failurePicturePath;
break;
case self::TEST_RESULT_COLOR_GREEN:
return $this->_successPicturePath;
break;
default:
return $this->_incompletePicturePath;
}
}
/**
* @return boolean
*/
private function _isGreenTestResult()
{
return count($this->_errors) === 0 &&
count($this->_failures) === 0 &&
count($this->_incompletes) === 0 &&
count($this->_skips) === 0;
}
/**
* @return boolean
*/
private function _isRedTestResult()
{
return count($this->_errors) > 0 ||
count($this->_failures) > 0;
}
/**
* @return boolean
*/
private function _isYellowTestResult()
{
return count($this->_errors) === 0 &&
count($this->_failures) === 0 &&
(count($this->_incompletes) > 0 ||
count($this->_skips) > 0);
}
}Hooking the Growl test listener into the PHPUnit ecosystem
To make use of the just outlined test listener it's necessary to add an entry to PHPUnit's XML configuration file telling PHPUnit which test listener class to utilize and where it's located in the file system. In a next step the images for the three possible Growl notifications have to be added to the local file system, and as the Growl test listener constructor takes these as arguments they have also to be injected in the PHPUnit XML configuration file (i.e. phpunit-offline.xml). Take a peek yourself how this is done in the next listing.<phpunit backupGlobals="false"
backupStaticAttributes="true"
bootstrap="bootstrap.php"
colors="true"
convertErrorsToExceptions="true"
convertNoticesToExceptions="true"
convertWarningsToExceptions="true"
processIsolation="true"
stopOnFailure="true"
syntaxCheck="true"
testSuiteLoaderClass="PHPUnit_Runner_StandardTestSuiteLoader">
<testsuites>
<testsuite name="Zend_Service_GitHub Offline Testsuite">
<directory>Zend/Service/GitHub</directory>
<directory>Zend/Service</directory>
</testsuite>
</testsuites>
<groups>
<include>
<group>offline</group>
</include>
</groups>
<listeners>
<listener class="PHPUnit_Extensions_TestListener_GrowlTestListener"
file="/Users/stolt/Work/GrowlTestListener.php">
<arguments>
<string>$HOME/Pictures/pass.png</string>
<string>$HOME/Pictures/pending.png</string>
<string>$HOME/Pictures/fail.png</string>
</arguments>
</listener>
</listeners>
</phpunit>Putting the Growl test listener to work
Attention shameless plug! As an example application for a continuous testing session I chose a Zend Framework Service component I'm currently working on. To set up the continuously testing workflow, stakeout.rb is still my #1 choice, but in a recent blog post Andy Stanberry shows another tool dubbed Kicker which seems to be coequal. The following console snippet shows in a concrete scenario how to utilize stakeout.rb to watch for any changes on the Zend_Service_GitHub component or it's backing tests which immediately trigger the test suite execution if one is detected.stakeout.rb 'phpunit --configuration phpunit-offline.xml' **/*.{php} ../Zend/**/*.php ../Zend/Service/**/*.php
In the classic TDD cycle we start with a failing test. Creating the test, adding the assertions that the system under test (SUT) has to fulfill and saving the according test class automatically triggers the test suite execution which ends up in the next shown Growl notification.
Nest a très important client call comes in and since we are clever, a quick TextMate shortcut marks the currently worked on test as incomplete. This step might be a bit controversy as it's also suggested to leave the last worked on test broken, but I got to show you the pending/incomplete Growl notification ;D
After finishing the 'interruptive' client call aka context switch we can continue to work on the feature of the SUT until it fulfills the expected behavior which will be radiated via the next shown Growl notification. Happy Growl flavored testing!
* As you might notice in the shown Growl notification images there's a test suite count of 9 while we are only operating on a single one, this seems to be a possible PHPUnit bug, or just a misconfiguration of my testing environment.
In case you got a solution for this problem feel free to add an illuminating comment.
Installing the PHP redis extension on Mac OS X 14 May 2010 9:53 PM (14 years ago)
Recently I took a look at Redis, a popular and advanced key-value store. Peeking at the supported languages section of the project's website you'll notice a lot of client libraries available for PHP. Two out of them caught my particular attention: Rediska due to it's impressive Zend Framework integration and phpredis as it's a native PHP extension written in C and therefore supposed to be blazingly faster than vanilla PHP client libraries. The following blog post will show how to install and configure the aforementioned, native PHP extension on a Mac OS X system.
The next steps assume that you've installed redis on your machine. In case you are using MacPorts and haven't installed the key-value store yet, all it takes are the following two commands and you're good to go. In case you prefer Homebrew for managing your package/software installations, there's also a Formula for redis available that allows you to install it via brew install redis.
sudo port install redis sudo launchctl load -w /Library/LaunchDaemons/org.macports.redis.plistThe very first step for building the native PHP redis extension is to get the source code by cloning the GitHub repository of the extension without it's history revisions.
mkdir phpredis-build cd phpredis-build git clone --depth 1 git://github.com/nicolasff/phpredis.git cd phpredisThe next task is to compile the extension with the following batch of commands.
phpize ./configure make sudo make installThe next to last step is to alternate your php.ini, use php --ini | grep 'Loaded' to get the location of it on your system, so that the redis module/extension is available to your PHP ecosystem. Therefor simply add extension=redis.so in the Dynamic Extensions section of your php.ini. Afterwards you can verify that the redis module is loaded and available via one of the following commands.
php -m | grep redis php -i | grep 'Redis Support'To make the extension also available to the running Apache PHP module you'll need to restart the Apache server. Looking at phpinfo()'s output in a browser you should see the entry shown in the next image.
For testing the communication between the just installed redis extension and the running Redis server, I further created a simple test script called redis-glue-test.php you can fetch from GitHub and run via the next commands.
curl -s http://gist.github.com/raw/402018/redis-glue-test.php -o redis-glue-test.php php redis-glue-test.phpWhen you see the following shown console output you're good to go. Happy Redising!
%3C/em%3E's%20output%20in%20a%20browser%20you%20should%20see%20the%20entry%20shown%20in%20the%20next%20image.%3Cbr%20/%3E%0A%3Cbr%20/%3E%0A%3Ca%20href%3D%22http://www.flickr.com/photos/raphaelstolt/4607738861/%22%20title%3D%22Enabled%20redis%20extension%20by%20Raphael%20Stolt,%20on%20Flickr%22%3E%3Cimg%20src%3D%22http://farm4.static.flickr.com/3320/4607738861_d24901cd7d_o.gif%22%20width%3D%22608%22%20height%3D%22199%22%20border%3D%220%22%20alt%3D%22Enabled%20redis%20extension%22%20/%3E%3C/a%3E%3Cbr%20/%3E%0A%3Cbr%20/%3E%0AFor%20testing%20the%20communication%20between%20the%20just%20installed%20redis%20extension%20and%20the%20running%20Redis%20server,%20I%20further%20created%20a%20simple%20test%20script%20called%20%3Ca%20href%3D%22http://gist.github.com/402018%22%20target%3D%22_self%22%3Eredis-glue-test.php%3C/a%3E%20you%20can%20fetch%20from%20GitHub%20and%20run%20via%20the%20next%20commands.%3Cpre%3Ecurl%20-s%20http://gist.github.com/raw/402018/redis-glue-test.php%20-o%20redis-glue-test.php%0Aphp%20redis-glue-test.php%3C/pre%3EWhen%20you%20see%20the%20following%20shown%20console%20output%20you're%20good%20to%20go.%20Happy%20Redising!%3Cbr%20/%3E%0A%3Cbr%20/%3E%0A%3Ca%20href%3D%22http://www.flickr.com/photos/raphaelstolt/4608312060/%22%20title%3D%22Output%20of%20redis-glue-test.php%20by%20Raphael%20Stolt,%20on%20Flickr%22%3E%3Cimg%20src%3D%22http://farm2.static.flickr.com/1295/4608312060_9de6daae3f_o.gif%22%20width%3D%22627%22%20height%3D%22304%22%20border%3D%220%22%20alt%3D%22Output%20of%20redis-glue-test.php%22%20/%3E%3C/a%3E)
Using MongoHq in Zend Framework based applications 16 Mar 2010 1:32 PM (15 years ago)
As the name slightly foreshadows MongoHq is a currently bit pricey cloud-based hosting solution for MongoDb databases provided by CommonThread. Since they went live a few weeks ago I signed up for the small plan and started to successfully re-thinker with it in an exploratory Zend Framework based application.
Therefore the following post will show how to bootstrap such an instance into a Zend Framework based application and how to use it from there in some simple scenarios like storing data coming from a Zend_Form into a designated collection and vice versa fetching it from there.
Bootstrapping a MongoHq enabled connection
To establish and make the MongoDb connection application-wide available the almighty Zend_Application component came to the rescue again. After reading Matthew Weier O'Phinney's enlightening blog post about creating re-usable Zend_Application resource plugins and deciding to use MongoDb in some more exploratory projects, I figured it would be best to create such a plugin and ditch the also possible resource method approach.The next code listing shows a possible implementation of the MongoDb resource plugin initializing a Mongo instance for the given APPLICATION_ENV (i.e. production) mode.
For the other application environment modes (development | testing | staging) it's currently assumed that no database authentication is enabled, which is also the default when using MongoDb, so you might need to adapt the plugin to your differing needs; and since I'm currently only rolling on the small plan the support for multiple databases is also not accounted for.
library/Recordshelf/Resource/MongoDb.php
<?phpWith the MongoDb resource plugin in the place to be, it's time to make it known to the boostrapping mechanism which is done by registering the resource plugin in the application.ini.
class Recordshelf_Resource_MongoDb
extends Zend_Application_Resource_ResourceAbstract
{
/**
* Definable Mongo options.
*
* @var array
*/
protected $_options = array(
'hostname' => '127.0.0.1',
'port' => '27017',
'username' => null,
'password' => null,
'databasename' => null,
'connect' => true
);
/**
* Initalizes a Mongo instance.
*
* @return Mongo
* @throws Zend_Exception
*/
public function init()
{
$options = $this->getOptions();
if (null !== $options['username'] &&
null !== $options['password'] &&
null !== $options['databasename'] &&
'production' === APPLICATION_ENV) {
// Database Dns with MongoHq credentials
$mongoDns = sprintf('mongodb://%s:%s@%s:%s/%s',
$options['username'],
$options['password'],
$options['hostname'],
$options['port'],
$options['databasename']
);
} elseif ('production' !== APPLICATION_ENV) {
$mongoDns = sprintf('mongodb://%s:%s/%s',
$options['hostname'],
$options['port'],
$options['databasename']
);
} else {
$exceptionMessage = sprintf(
'Recource %s is not configured correctly',
__CLASS__
);
throw new Zend_Exception($exceptionMessage);
}
try {
return new Mongo($mongoDns, array('connect' => $options['connect']));
} catch (MongoConnectionException $e) {
throw new Zend_Exception($e->getMessage());
}
}
}
Further the MongoHq credentials, which are available in the MongoHq > My Database section, and the main database name are added to the configuration file which will be used to set the definable resource plugin ($_)options and to connect to the hosted database.
application/configs/application.ini
[production]
pluginPaths.Recordshelf_Resource = "Recordshelf/Resource"
resources.mongodb.username = __MONGOHQ_USERNAME__
resources.mongodb.password = __MONGOHQ_PASSWORD__
resources.mongodb.hostname = __MONGOHQ_HOSTNAME__
resources.mongodb.port = __MONGOHQ_PORT__
resources.mongodb.databasename = __MONGOHQ_DATABASENAME__
...
Cloudifying documents into collections
Having the MongoHq enabled connection in the bootstrapping mechanism it can now be picked up from there and used in any Zend Framework application context.The example action method (i.e. proposeAction) assumes data (i.e. a tech talk proposal to revive the example domain from my last blog post) coming from a Zend_Form which will be stored in a collection named proposals, a table in old relational database think.
The next code listings states the action method innards to do so by injecting the valid form values into a model class which provides accessors and mutators for the domain model's properties and can transform them into a proposal document aka an array structure.
application/controllers/ProposalController.php
<?phpNext the model/data mappper is initialized, which triggers the picking of the MongoHq enabled Mongo connection instance and the auto-determination of the collection name to use based on the mapper's class name. Subsequently the populated model instance is passed into the mappper's insert method which is pulling the document (array structure) and doing the actual insert into the proposals collection.
class ProposalController extends Zend_Controller_Action
{
public function indexAction()
{
$this->view->form = new Recordshelf_Form_Proposal();
}
public function thanksAction()
{
}
public function proposeAction()
{
$this->_helper->viewRenderer->setNoRender();
$form = new Recordshelf_Form_Proposal();
$request = $this->getRequest();
if ($this->getRequest()->isPost()) {
if ($form->isValid($request->getPost())) {
$model = new Recordshelf_Model_Proposal($form->getValues());
$mapper = new Recordshelf_Model_ProposalMapper();
if ($mapper->insert($model)) {
return $this->_helper->redirector('thanks');
}
$this->view->form = $form;
return $this->render('index');
} else {
$this->view->form = $form;
return $this->render('index');
}
}
}
}
To give you an idea of the actual document structure it's shown in the next listing, followed by the model/data mapper implementation.
Array
(
[state] => new
[created] => MongoDate Object
(
[sec] => 1268774242
[usec] => 360831
)
[submitee] => Array
(
[title] => Mr
[firstname] => John
[familyname] => Doe
[email] => john.doe@gmail.com
[twitter] => johndoe
)
[title] => How to get a real name
[description] => Some descriptive text...
[topictags] => Array
(
[0] => John
[1] => Doe
[2] => Anonymous
)
)
application/models/ProposalMapper.php
<?php
class Recordshelf_Model_ProposalMapper
{
private $_mongo;
private $_collection;
private $_databaseName;
private $_collectionName;
public function __construct()
{
$frontController = Zend_Controller_Front::getInstance();
$this->_mongo = $frontController->getParam('bootstrap')
->getResource('mongoDb');
$config = $frontController->getParam('bootstrap')
->getResource('config');
$this->_databaseName = $config->resources->mongodb->get('databasename');
$replaceableClassNameparts = array(
'recordshelf_model_',
'mapper'
);
$this->_collectionName = str_replace($replaceableClassNameparts, '',
strtolower(__CLASS__) . 's');
$this->_collection = $this->_mongo->selectCollection(
$this->_databaseName,
$this->_collectionName
);
}
/**
* Inserts a proposal document/model into the proposals collection.
*
* @param Recordshelf_Model_Proposal $proposal The proposal document/model.
* @return MongoId
* @throws Zend_Exception
*/
public function insert(Recordshelf_Model_Proposal $proposal)
{
$proposalDocument = $proposal->getValues();
try {
if ($this->_collection->insert($proposalDocument, true)) {
return $proposalDocument['_id'];
}
} catch (MongoCursorException $mce) {
throw new Zend_Exception($mce->getMessage());
}
}
}
Querying and retrieving the cloudified data
As what comes in must come out, the next interaction with the Document Database Management System (DocDBMS) is about retrieving some afore-stored talk proposal documents from the collection so they can be rendered to the application's user. This isn't really MongoHq specific anymore, like most of the previous model parts, and is just here to round up this blog post and use some more of that MongoDb goodness. Looks like I have to look for an anonymous self-help group that stuff is highly addictive.Anyway the next listing shows the action method fetching all stored documents available in the proposals collection. To save some CO2 on this blog post all documents are fetched, which ends up in the most trivial query but as you can figure the example domain provides a bunch of query examples like only proposals for a given topic tag, specific talk title or a given proposal state which can be easily created via passed-through Http request parameters.
application/controllers/ProposalController.php
<?phpThe last code listing shows the above used fetchAll method of the data mapper class returning an array of stored proposal documents mapped to their domain model (i.e. Recordshelf_Model_Proposal) in the application.
class ProposalController extends Zend_Controller_Action
{
...
public function listAction()
{
$mapper = new Recordshelf_Model_ProposalMapper();
$proposals = $mapper->fetchAll();
// For iterating the Recordshelf_Model_Proposal's in the view
$this->view->proposals = $proposals;
}
}
application/models/ProposalMapper.php
<?php
class Recordshelf_Model_ProposalMapper
{
...
/**
* Fetches all stored talk proposals.
*
* @return array
*/
public function fetchAll()
{
$cursor = $this->_collection->find();
$proposals = array();
foreach ($cursor as $documents) {
$proposal = new Recordshelf_Model_Proposal();
foreach ($documents as $property => $value) {
if ('submitee' === $property) {
$proposal->submitee = new Recordshelf_Model_Submitee($value);
} else {
$proposal->$property = $value;
}
}
$proposals[] = $proposal;
}
return $proposals;
}
}
Utilizing Twitter lists with Zend_Service_Twitter 4 Feb 2010 4:05 PM (15 years ago)
Several months ago Twitter added the list feature to it's public API. While debating some use cases for an event registration application I stumbled upon an interesting feature, which adds participants automatically to a Twitter list upon registration. This way registered and interested users can discover like-minded individuals and get in touch prior to any pre-social event activities. This post will show how this feature can be implemented by utilizing the Zend_Service_Twitter component, and how it then can be used in a Zend Framework based application.
Implementing the common list features
Looking at the three relevant parts of the Twitter list API some common features emerged and had to be supported to get the feature out of the door. These are namely the creation, deletion of new lists and the addition, removal of list members (i.e. event participants). Since the current Twitter component doesn't support these list operations out of the box it was time to put that develeoper hat on and get loose; which was actually a joy due to the elegance of the extended Zend_Service_Twitter component laying all the groundwork.A non-feature-complete implementation is shown in the next code listing and can alternatively be pulled from GitHub. Currently it only supports the above stated common operations plus the ability to get the lists of a Twitter account and it's associated members; but feel free to fork it or even turn it into an official proposal.
<?php
require_once 'Zend/Service/Twitter.php';
require_once 'Zend/Service/Twitter/Exception.php';
class Recordshelf_Service_Twitter_List extends Zend_Service_Twitter
{
const LIST_MEMBER_LIMIT = 500;
const MAX_LIST_NAME_LENGTH = 25;
const MAX_LIST_DESCRIPTION_LENGTH = 100;
/**
* Initializes the service and adds the list to the method types
* of the parent service class.
*
* @param string $username The Twitter account name.
* @param string $password The Twitter account password.
* @see Zend_Service_Twitter::_methodTypes
*/
public function __construct($username = null, $password = null)
{
parent::__construct($username, $password);
$this->_methodTypes[] = 'list';
}
/**
* Creates a list associated to the current user.
*
* @param string $listname The listname to create.
* @param array $options The options to set whilst creating the list.
* Allows to set the list creation mode (public|private)
* and the list description.
* @return Zend_Rest_Client_Result
* @throws Zend_Service_Twitter_Exception
*/
public function create($listname, array $options = array())
{
$this->_init();
if ($this->_existsListAlready($listname)) {
$exceptionMessage = 'List with name %s exists already';
$exceptionMessage = sprintf($exceptionMessage, $listname);
throw new Zend_Service_Twitter_Exception($exceptionMessage);
}
$_options = array('name' => $this->_validListname($listname));
foreach ($options as $key => $value) {
switch (strtolower($key)) {
case 'mode':
$_options['mode'] = $this->_validMode($value);
break;
case 'description':
$_options['description'] = $this->_validDescription($value);
break;
default:
break;
}
}
$path = '/1/%s/lists.xml';
$path = sprintf($path, $this->getUsername());
$response = $this->_post($path, $_options);
return new Zend_Rest_Client_Result($response->getBody());
}
/**
* Deletes an owned list of the current user.
*
* @param string $listname The listname to delete.
* @return Zend_Rest_Client_Result
* @throws Zend_Service_Twitter_Exception
*/
public function delete($listname)
{
$this->_init();
if (!$this->_isListAssociatedWithUser($listname)) {
$exceptionMessage = 'List %s is not associate with user %s ';
$exceptionMessage = sprintf($exceptionMessage,
$listname,
$this->getUsername()
);
throw new Zend_Service_Twitter_Exception($exceptionMessage);
}
$_options['_method'] = 'DELETE';
$path = '/1/%s/lists/%s.xml';
$path = sprintf($path,
$this->getUsername(),
$this->_validListname($listname)
);
$response = $this->_post($path, $_options);
return new Zend_Rest_Client_Result($response->getBody());
}
/**
* Adds a member to a list of the current user.
*
* @param integer $userId The numeric user id of the member to add.
* @param string $listname The listname to add the member to.
* @return Zend_Rest_Client_Result
* @throws Zend_Service_Twitter_Exception
*/
public function addMember($userId, $listname)
{
$this->_init();
if (!$this->_isListAssociatedWithUser($listname)) {
$exceptionMessage = 'List %s is not associate with user %s ';
$exceptionMessage = sprintf($exceptionMessage,
$listname,
$this->getUsername()
);
throw new Zend_Service_Twitter_Exception($exceptionMessage);
}
$_options['id'] = $this->_validInteger($userId);
$path = '/1/%s/%s/members.xml';
$path = sprintf($path,
$this->getUsername(),
$this->_validListname($listname)
);
if ($this->_isListMemberLimitReached($listname)) {
$exceptionMessage = 'List can contain no more than %d members';
$exceptionMessage = sprintf($exceptionMessage,
self::LIST_MEMBER_LIMIT
);
throw new Zend_Service_Twitter_Exception($exceptionMessage);
}
$response = $this->_post($path, $_options);
return new Zend_Rest_Client_Result($response->getBody());
}
/**
* Removes a member from a list of the current user.
*
* @param integer $userId The numeric user id of the member to remove.
* @param string $listname The listname to remove the member from.
* @return Zend_Rest_Client_Result
* @throws Zend_Service_Twitter_Exception
*/
public function removeMember($userId, $listname)
{
$this->_init();
if (!$this->_isListAssociatedWithUser($listname)) {
$exceptionMessage = 'List %s is not associate with user %s ';
$exceptionMessage = sprintf($exceptionMessage,
$listname,
$this->getUsername()
);
throw new Zend_Service_Twitter_Exception($exceptionMessage);
}
$_options['_method'] = 'DELETE';
$_options['id'] = $this->_validInteger($userId);
$path = '/1/%s/%s/members.xml';
$path = sprintf($path,
$this->getUsername(),
$this->_validListname($listname)
);
$response = $this->_post($path, $_options);
return new Zend_Rest_Client_Result($response->getBody());
}
/**
* Fetches the list members of the current user.
*
* @param string $listname The listname to fetch members from.
* @return Zend_Rest_Client_Result
* @throws Zend_Service_Twitter_Exception
*/
public function getMembers($listname) {
$this->_init();
$path = '/1/%s/%s/members.xml';
$path = sprintf($path,
$this->getUsername(),
$this->_validListname($listname)
);
$response = $this->_get($path);
return new Zend_Rest_Client_Result($response->getBody());
}
/**
* Fetches the list of the current user or any given user.
*
* @param string $username The username of the list owner.
* @return Zend_Rest_Client_Result
*/
public function getLists($username = null)
{
$this->_init();
$path = '/1/%s/lists.xml';
if (is_null($username)) {
$path = sprintf($path, $this->getUsername());
} else {
$path = sprintf($path, $username);
}
$response = $this->_get($path);
return new Zend_Rest_Client_Result($response->getBody());
}
/**
* Checks if the list exists already to avoid number
* indexed recreations.
*
* @param string $listname The list name.
* @return boolean
* @throws Zend_Service_Twitter_Exception
*/
private function _existsListAlready($listname)
{
$_listname = $this->_validListname($listname);
$lists = $this->getLists();
$_lists = $lists->lists;
foreach ($_lists->list as $list) {
if ($list->name == $_listname) {
return true;
}
}
return false;
}
/**
* Checks if the list is associated with the current user.
*
* @param string $listname The list name.
* @return boolean
*/
private function _isListAssociatedWithUser($listname)
{
return $this->_existsListAlready($listname);
}
/**
* Checks if the list member limit is reached.
*
* @param string $listname The list name.
* @return boolean
*/
private function _isListMemberLimitReached($listname)
{
$members = $this->getMembers($listname);
return self::LIST_MEMBER_LIMIT < count($members->users->user);
}
/**
* Returns the list creation mode or returns the private mode when invalid.
* Valid values are private or public.
*
* @param string $creationMode The list creation mode.
* @return string
*/
private function _validMode($creationMode)
{
if (in_array($creationMode, array('private', 'public'))) {
return $creationMode;
}
return 'private';
}
/**
* Returns the list name or throws an Exception when invalid.
*
* @param string $listname The list name.
* @return string
* @throws Zend_Service_Twitter_Exception
*/
private function _validListname($listname)
{
$len = iconv_strlen(trim($listname), 'UTF-8');
if (0 == $len) {
$exceptionMessage = 'List name must contain at least one character';
throw new Zend_Service_Twitter_Exception($exceptionMessage);
} elseif (self::MAX_LIST_NAME_LENGTH < $len) {
$exceptionMessage = 'List name must contain no more than %d characters';
$exceptionMessage = sprintf($exceptionMessage,
self::MAX_LIST_NAME_LENGTH
);
throw new Zend_Service_Twitter_Exception($exceptionMessage);
}
return trim($listname);
}
/**
* Returns the list description or throws an Exception when invalid.
*
* @param string $description The list description.
* @return string
* @throws Zend_Service_Twitter_Exception
*/
private function _validDescription($description)
{
$len = iconv_strlen(trim($description), 'UTF-8');
if (0 == $len) {
return '';
} elseif (self::MAX_LIST_DESCRIPTION_LENGTH < $len) {
$exceptionMessage = 'List description must contain no more than %d characters';
$exceptionMessage = sprintf($exceptionMessage,
self::MAX_LIST_DESCRIPTION_LENGTH
);
throw new Zend_Service_Twitter_Exception($exceptionMessage);
}
return trim(strip_tags($description));
}
}
Adding the 'auto list' feature
For using the above implemented add member feature it's assumed that a participant has provided a valid and existing Twitter username, his approval of being added to the event list (i.e. zfweekend) and that he further has been registered effectively. To have the name of the Twitter list to act on and the account credentials available corresponding configuration entries are set as shown next.application/configs/application.ini
With the Twitter credentials and the list name available it's now possible to pull this feature into the register method of the register Action Controller, where it's applied as shown in the outro listing. As you will see, besides some bad practices due to demonstration purposes, the register Form makes use of a custom TwitterScreenName validator and filter which are also available via GitHub. Happy Twitter listing!
[production]
twitter.username = __USERNAME__
twitter.password = __PASSWORD__
twitter.auto.listname = zfweekend
<?php
class RegisterController extends Zend_Controller_Action
{
/**
* @badpractice Push this into a specific Form class.
* @return Zend_Form
*/
private function _getForm()
{
$form = new Zend_Form();
$form->setAction('/register/register')
->setMethod('post');
$twitterScreenName = $form->createElement('text', 'twitter_screen_name',
array('label' => 'Twittername: ')
);
$twitterScreenName->addValidator(new Recordshelf_Validate_TwitterScreenName())
->setRequired(true)
->setAllowEmpty(false)
->addFilter(new Recordshelf_Filter_TwitterScreenName());
$autoListApproval = $form->createElement('checkbox', 'auto_list_approval',
array('label' => 'I approved to be added to the event Twitter list: ')
);
$form->addElement($twitterScreenName)
->addElement($autoListApproval)
->addElement('submit', 'register', array('label' => ' Register '));
return $form;
}
public function indexAction()
{
$this->view->form = $this->_getForm();
}
public function thanksAction()
{
}
/**
* @badpractice Handle possible Exception of
* Recordshelf_Service_Twitter_List::addMember.
* @return Recordshelf_Service_Twitter_List
*/
public function registerAction()
{
$this->_helper->viewRenderer->setNoRender();
$form = $this->_getForm();
$request = $this->getRequest();
if ($this->getRequest()->isPost()) {
if ($form->isValid($request->getPost())) {
$model = new Recordshelf_Model_Participant($form->getValues());
$model->save();
if ($form->getElement('auto_list_approval')->isChecked()) {
$twitterScreenName = $form->getValue('twitter_screen_name');
$twitter = $this->_getTwitterListService();
$response = $twitter->user->show($twitterScreenName);
$userId = (string) $response->id;
$response = $twitter->list->addMember($userId,
$this->_getTwitterListName());
$model->hasBeenAddedToTwitterList(true);
$model->update();
return $this->_helper->redirector('thanks');
}
} else {
return $this->_helper->redirector('index');
}
}
}
/**
* @badpractice Push this into a dedicated Helper or something similar.
* @return Recordshelf_Service_Twitter_List
*/
private function _getTwitterListService()
{
$config = Zend_Registry::get('config');
return new Recordshelf_Service_Twitter_List(
$config->twitter->get('username'),
$config->twitter->get('password')
);
}
/**
* @badpractice Push this into a dedicated Helper or something similar.
* @return string
*/
private function _getTwitterListName()
{
$config = Zend_Registry::get('config');
return $config->twitter->auto->get('listname');
}
}
Closing and reopening GitHub issues via PHPUnit tests 19 Jan 2010 12:41 PM (15 years ago)
Since PHPUnit 3.4.0 a new extension point for interacting with issue tracking systems (TTS) based on the test results has been added to PHP's first choice xUnit framework. The extension point has been introduced by an abstract PHPUnit_Extensions_TicketListener class, which allows developer to add tailor-made ticket listeners supporting their favoured TTS. Currently PHPUnit ships with a single ticket listener for Trac as it's still the used TTS for the framework itself. As I start to become more and more accustomed to use GitHub for some of my exploratory projects and hacks, the following blog post will contain a GitHub_TicketListener implementation and a showcase of it's usage.
Annotating tests with ticket meta data
As you might know, it's considered to be a best practice to write a test for each new ticket representing a bug and drive the system under test (SUT) till the issue is resolved. This extension of test-driven development is also known as test-driven bug fixing. To create a relation between these tests and their associated tickets, PHPUnit provides a new @ticket annotation which will be analyzed before each test is run. The following code listing shows such an annotated test.<?php
require_once 'PHPUnit/Framework.php';
class ExampleTest extends PHPUnit_Framework_TestCase
{
....
/**
* @ticket 2
* @test
*/
public function shouldGuarantyThatTheSutHandlesTheIssueCorrectly()
{
// test code
}
....
Peeking at the GitHub_TicketListener implementation
The current version (3.4.6) of PHPUnit has a pending issue regarding the abstract TicketListener class, so the first step is to apply an 'exploratory' patch, which might break the functionality of the shipped Trac ticket listener but will enable the use of the one for GitHub's TTS.The next step en route to a working GitHub_TicketListener is to extend the patched abstract PHPUnit_Extensions_TicketListener class. This abstract class contains two abstract methods named getTicketInfo and updateTicket which have to be implemented by the specific ticket listener class, and will be responsible for the interaction with the TTS.
The implementation of the getTicketInfo method retrieves the ticket status for the annotated ticket, while the updateTicket method is responsible for changing the ticket status based on the test result and the former ticket state. Both implementations make use of the relevant TTS part of the GitHub API by utilizing PHP's curl extension as shown in the next code listing which alternatively is available via this gist.
<?php
require_once('PHPUnit/Extensions/TicketListener.php');
PHPUnit_Util_Filter::addFileToFilter(__FILE__, 'PHPUNIT');
/**
* A ticket listener that interacts with the GitHub issue API.
*/
class PHPUnit_Extensions_TicketListener_GitHub extends
PHPUnit_Extensions_TicketListener
{
const STATUS_CLOSE = 'closed';
const STATUS_REOPEN = 'reopened';
private $_username = null;
private $_apiToken = null;
private $_repository = null;
private $_apiPath = null;
private $_printTicketStateChanges = false;
/**
* @param string $username The username associated with the GitHub account.
* @param string $apiToken The API token associated with the GitHub account.
* @param string $repository The repository of the system under test (SUT) on GitHub.
* @param string $printTicketChanges Boolean flag to print the ticket state
* changes in the test result.
* @throws RuntimeException
*/
public function __construct($username, $apiToken, $repository,
$printTicketStateChanges = false)
{
if ($this->_isCurlAvailable() === false) {
throw new RuntimeException('The dependent curl extension is not available');
}
if ($this->_isJsonAvailable() === false) {
throw new RuntimeException('The dependent json extension is not available');
}
$this->_username = $username;
$this->_apiToken = $apiToken;
$this->_repository = $repository;
$this->_apiPath = 'http://github.com/api/v2/json/issues';
$this->_printTicketStateChanges = $printTicketStateChanges;
}
/**
* @param integer $ticketId
* @return string
* @throws PHPUnit_Framework_Exception
*/
public function getTicketInfo($ticketId = null)
{
if (!ctype_digit($ticketId)) {
return $ticketInfo = array('status' => 'invalid_ticket_id');
}
$ticketInfo = array();
$apiEndpoint = "{$this->_apiPath}/show/{$this->_username}/"
. "{$this->_repository}/{$ticketId}";
$issueProperties = $this->_callGitHubIssueApiWithEndpoint($apiEndpoint, true);
if ($issueProperties['state'] === 'open') {
return $ticketInfo = array('status' => 'new');
} elseif ($issueProperties['state'] === 'closed') {
return $ticketInfo = array('status' => 'closed');
} elseif ($issueProperties['state'] === 'unknown_ticket') {
return $ticketInfo = array('status' => $issueProperties['state']);
}
}
/**
* @param string $ticketId The ticket number of the ticket under test (TUT).
* @param string $statusToBe The status of the TUT after running the associated test.
* @param string $message The additional message for the TUT.
* @param string $resolution The resolution for the TUT.
* @throws PHPUnit_Framework_Exception
*/
protected function updateTicket($ticketId, $statusToBe, $message, $resolution)
{
$apiEndpoint = null;
$acceptedResponseIssueStates = array('open', 'closed');
if ($statusToBe === self::STATUS_CLOSE) {
$apiEndpoint = "{$this->_apiPath}/close/{$this->_username}/"
. "{$this->_repository}/{$ticketId}";
} elseif ($statusToBe === self::STATUS_REOPEN) {
$apiEndpoint = "{$this->_apiPath}/reopen/{$this->_username}/"
. "{$this->_repository}/{$ticketId}";
}
if (!is_null($apiEndpoint)) {
$issueProperties = $this->_callGitHubIssueApiWithEndpoint($apiEndpoint);
if (!in_array($issueProperties['state'], $acceptedResponseIssueStates)) {
throw new PHPUnit_Framework_Exception(
'Recieved an unaccepted issue state from the GitHub Api');
}
if ($this->_printTicketStateChanges) {
printf("\nUpdating GitHub issue #%d, status: %s\n", $ticketId,
$statusToBe);
}
}
}
/**
* @return boolean
*/
private function _isCurlAvailable()
{
return extension_loaded('curl');
}
/**
* @return boolean
*/
private function _isJsonAvailable()
{
return extension_loaded('json');
}
/**
* @param string $apiEndpoint API endpoint to call against the GitHub issue API.
* @param boolean $isShowMethodCall Show method of the GitHub issue API is called?
* @return array
* @throws PHPUnit_Framework_Exception
*/
private function _callGitHubIssueApiWithEndpoint($apiEndpoint,
$isShowMethodCall = false)
{
$curlHandle = curl_init();
curl_setopt($curlHandle, CURLOPT_URL, $apiEndpoint);
curl_setopt($curlHandle, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($curlHandle, CURLOPT_FAILONERROR, true);
curl_setopt($curlHandle, CURLOPT_FRESH_CONNECT, true);
curl_setopt($curlHandle, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curlHandle, CURLOPT_HTTPPROXYTUNNEL, true);
curl_setopt($curlHandle, CURLOPT_USERAGENT, __CLASS__);
curl_setopt($curlHandle, CURLOPT_POSTFIELDS,
"login={$this->_username}&token={$this->_apiToken}");
$response = curl_exec($curlHandle);
// Unknown tickets throw a 403 error
if (!$response && $isGetTicketInfoCall) {
return array('state' => 'unknown_ticket');
}
if (!$response) {
$curlErrorMessage = curl_error($curlHandle);
$exceptionMessage = "A failure occured while talking to the "
. "GitHub issue Api. {$curlErrorMessage}.";
throw new PHPUnit_Framework_Exception($exceptionMessage);
}
$issue = (array) json_decode($response);
$issueProperties = (array) $issue['issue'];
curl_close($curlHandle);
return $issueProperties;
}
}
Plugging the GitHub_TicketListener into the PHPUnit test environment
To hook the GitHub ticket listener into the test runtime environment PHPUnit provides several approaches to do so. The chosen approach makes use of a XML configuration file which allows an injection of the ticket listener in a declarative manner. As you will see in the configuration file snippet, the GitHub ticket listener is initialized with four parameters: The first one is the GitHub username, followed by the GitHub API token, the associated GitHub project, and a boolean flag for displaying the ticket status changes in the test result.<phpunit>To run the tests against a SUT and see the PHPUnit GitHub TTS interaction at work, all it takes is the forthcoming PHPUnit Cli call.
<listeners>
<listener class="PHPUnit_Extensions_TicketListener_GitHub"
file="/path/to/GitHubTicketListener.php">
<arguments>
<string>raphaelstolt</string>
<string>API_TOKEN</string>
<string>PROJECT_NAME</string>
<boolean>true</boolean>
</arguments>
</listener>
</listeners>
</phpunit>
phpunit --configuration github-ticketlistener.xml ExampleTest.phpThe outro screenshot shows the test result for an example SUT along with a GitHub TTS interaction due to a passing test which is associated with a open ticket in the TTS.
A final note: As the interaction with an TTS adds some overhead to the test execution and thereby might cause Slow Tests, ticket listener should only be considered in non time-critical test scenarios (e.g. nightly builds).
Zend Framework 1.8 Web Application Development book review 14 Oct 2009 1:22 PM (15 years ago)
As the days are rapidly getting shorter, my reading appetite grows potentially and this evening I finished the 'Zend Framework 1.8 Web Application Development' book written by Keith Pope. While Keith worked on the book, I peeked several times at it's tutorial application, dubbed the Storefront, to get me going with the new Zend_Application component. Looking at it's code made me feel certain to get another great digest of the new features and components of version 1.8, and also a different practical perspective on web application development with the Zend Framework, once the book has been published. Therefor I got in touch with the publisher Packt and fortunately got a copy of which I'd like to share a personal review in this blog post.
What's in it?
The book opens with a quick run-through of the Model-View-Controller (MVC) architecture by creating a project structure via Zend_Tool and building a first very basic web application. While this introduction intentionally skips over a lot of details, the following chapter provides very detailed insights into the Zend Framework's MVC components by explaining the surrounded objects, the Design Patterns they are based upon and their interactions.After laying out that hefty block of theory the aforementioned tutorial application is introduced and built incrementally over several chapters; each one going into more detail for the specific application aspect. The highlight content of these chapters reach from introducing the Fat Model Skinny Controller concept, thoughts on Model design strategies which are reflected in a custom Storefront Model design, to developing application specific Front Controller Plugins, Action-Helpers, and View-Helpers. The application walk-through is completed by looking at general techniques to optimize the Storefront application and by building an automated PHPUnit Test Suite of functional tests utilizing Zend_Test to keep the Zend Framework based application self-reliant and refactorable.
Conclusion
The book by Keith Pope provides any interested PHP developer, who's not already sold on a specific framework, a thorough introduction to the vivid Zend Framework and it's use in a MVC based web application development context. The content of the book is delivered in a fluent, very enthusiastic and 'knowledge-pillowed' writing tone. By implementing or working through the Storefront application seasoned web developers using older versions of the Framework will get a good blue sheet on new components like Zend_Application and it's implication in the bootstrapping process; while new developers tending towards picking up the Zend Framework will get a current and well compiled guide, which might first start off with a steep learning-curve but will turn into profund knowledge once hanging in there.The only thing that seemed a bit odd to me, was the utilization of Ant instead of Phing as the build tool for the Storefront application to set the application environment, to remove all require_once statements from the framework library and to run the PHPUnit Test Suite; but this might also be inflicted by my Phing nuttiness.
%20architecture%20by%20creating%20a%20project%20structure%20via%20%3Ca%20href%3D%22http://framework.zend.com/manual/en/zend.tool.framework.html%22%20target%3D%22_self%22%3EZend_Tool%3C/a%3E%20and%20building%20a%20first%20very%20basic%20web%20application.%20While%20this%20introduction%20intentionally%20skips%20over%20a%20lot%20of%20details,%20the%20following%20chapter%20provides%20very%20detailed%20insights%20into%20the%20Zend%20Framework's%20MVC%20components%20by%20explaining%20the%20surrounded%20objects,%20the%20Design%20Patterns%20they%20are%20based%20upon%20and%20their%20interactions.%20%3Cbr%20/%3E%3Cbr%20/%3EAfter%20laying%20out%20that%20hefty%20block%20of%20theory%20the%20aforementioned%20tutorial%20application%20is%20introduced%20and%20built%20incrementally%20over%20several%20chapters;%20each%20one%20going%20into%20more%20detail%20for%20the%20specific%20application%20aspect.%20The%20highlight%20content%20of%20these%20chapters%20reach%20from%20introducing%20the%20Fat%20Model%20Skinny%20Controller%20concept,%20thoughts%20on%20Model%20design%20strategies%20which%20are%20reflected%20in%20a%20custom%20Storefront%20Model%20design,%20to%20developing%20application%20specific%20Front%20Controller%20Plugins,%20Action-Helpers,%20and%20View-Helpers.%20The%20application%20walk-through%20is%20completed%20by%20looking%20at%20general%20techniques%20to%20optimize%20the%20Storefront%20application%20and%20by%20building%20an%20automated%20%3Ca%20href%3D%22http://www.phpunit.de%22%20target%3D%22self%22%3EPHPUnit%3C/a%3E%20Test%20Suite%20of%20functional%20tests%20utilizing%20%3Ca%20href%3D%22http://framework.zend.com/manual/en/zend.test.html%22%20target%3D%22_self%22%3EZend_Test%3C/a%3E%20to%20keep%20the%20Zend%20Framework%20based%20application%20self-reliant%20and%20refactorable.%3Ch4%3EConclusion%3C/h4%3EThe%20book%20by%20Keith%20Pope%20provides%20any%20interested%20PHP%20developer,%20who's%20not%20already%20sold%20on%20a%20specific%20framework,%20a%20thorough%20introduction%20to%20the%20vivid%20Zend%20Framework%20and%20it's%20use%20in%20a%20MVC%20based%20web%20application%20development%20context.%20The%20content%20of%20the%20book%20is%20delivered%20in%20a%20fluent,%20very%20enthusiastic%20and%20'knowledge-pillowed'%20writing%20tone.%20By%20implementing%20or%20working%20through%20the%20Storefront%20application%20seasoned%20web%20developers%20using%20older%20versions%20of%20the%20Framework%20will%20get%20a%20good%20blue%20sheet%20on%20new%20components%20like%20%3Ca%20href%3D%22http://framework.zend.com/manual/en/zend.application.html%22%20target%3D%22_self%22%3EZend_Application%3C/a%3E%20and%20it's%20implication%20in%20the%20bootstrapping%20process;%20while%20new%20developers%20tending%20towards%20picking%20up%20the%20Zend%20Framework%20will%20get%20a%20current%20and%20well%20compiled%20guide,%20which%20might%20first%20start%20off%20with%20a%20steep%20learning-curve%20but%20will%20turn%20into%20profund%20knowledge%20once%20hanging%20in%20there.%3Cbr%20/%3E%3Cbr%20/%3EThe%20only%20thing%20that%20seemed%20a%20bit%20odd%20to%20me,%20was%20the%20utilization%20of%20%3Ca%20href%3D%22http://ant.apache.org%22%20target%3D%22_self%22%3EAnt%3C/a%3E%20instead%20of%20%3Ca%20href%3D%22http://phing.info%22%20target%3D%22_self%22%3EPhing%3C/a%3E%20as%20the%20build%20tool%20for%20the%20Storefront%20application%20to%20set%20the%20application%20environment,%20to%20remove%20all%20require_once%20statements%20from%20the%20framework%20library%20and%20to%20run%20the%20PHPUnit%20Test%20Suite;%20but%20this%20might%20also%20be%20inflicted%20by%20my%20Phing%20%3Ca%20href%3D%22http://raphaelstolt.blogspot.com/search/label/Phing%22%20target%3D%22_self%22%3Enuttiness%3C/a%3E.%3Cbr%20/%3E%3Cbr%20/%3E)
Logging to MongoDb and accessing log collections with Zend_Tool 19 Sep 2009 11:00 AM (15 years ago)
Influenced by a recent blog post of a colleague of mine and by being kind of broke on a Saturday night; I tinkered with the just recently discovered MongoDb and hooked it into the Zend_Log environment by creating a dedicated Zend_Log_Writer. The following post will therefore present a peek at a prototypesque implementation of this writer and show how the afterwards accumulated log entries can be accessed and filtered with a custom Zend_Tool project provider.
Logging to a MongoDb database
The following steps assume that an instance of a MongoDb server is running and that the required PHP MongoDb module is also installed and loaded. To by-pass log entries to a MongoDb database there is a need to craft a proper Zend_Log_Writer. This can be achieved by extending the Zend_Log_Writer_Abstract class, injecting a Mongo connection instance and implementing the actual write functionality as shown in the next listing.<?php
require_once 'Zend/Log/Writer/Abstract.php';
class Recordshelf_Log_Writer_MongoDb extends Zend_Log_Writer_Abstract
{
private $_db;
private $_connection;
/**
* @param Mongo $connection The MongoDb database connection
* @param string $db The MongoDb database name
* @param string $collection The collection name string the log entries
*/
public function __construct(Mongo $connection, $db, $collection)
{
$this->_connection = $connection;
$this->_db = $this->_connection->selectDB($db)->createCollection(
$collection
);
}
public function setFormatter($formatter)
{
require_once 'Zend/Log/Exception.php';
throw new Zend_Log_Exception(get_class() . ' does not support formatting');
}
public function shutdown()
{
$this->_db = null;
$this->_connection->close();
}
protected function _write($event)
{
$this->_db->insert($event);
}
/**
* Create a new instance of Recordshelf_Log_Writer_MongoDb
*
* @param array|Zen_Config $config
* @return Recordshelf_Log_Writer_MongoDb
* @throws Zend_Log_Exception
* @since Factory Interface available since release 1.10.0
*/
static public function factory($config)
{
$exceptionMessage = 'Recordshelf_Log_Writer_MongoDb does not currently '
. 'implement a factory';
throw new Zend_Exception($exceptionMessage);
}
}With the MongoDb writer available and added to the library directory of the application it's now possible to utilize this new storage backend as usual with the known Zend_Log component. The Mongo connection injected into the writer is configured via Zend_Config and initialized via the Zend_Application bootstrapping facility as shown in the listings below.application/configs/application.ini
[production] app.name = recordshelf .... log.mongodb.db = zf_mongo log.mongodb.collection = recordshelf_log log.mongodb.server = localhost log.priority = Zend_Log::CRIT ....
application/Bootstrap.php
<?php
class Bootstrap extends Zend_Application_Bootstrap_Bootstrap
{
protected $_logger;
protected function _initConfig()
{
Zend_Registry::set('config', new Zend_Config($this->getOptions()));
}
protected function _initLogger()
{
$this->bootstrap(array('frontController', 'config'));
$config = Zend_Registry::get('config');
$applicationName = $config->app->get('name', 'recordshelf');
$mongoDbServer = $config->log->mongodb->get('server', '127.0.0.1');
$mongoDbName = $config->log->mongodb->get('db', "{$applicationName}_logs");
$mongoDbCollection = $config->log->mongodb->get('collection', 'entries');
$logger = new Zend_Log();
$writer = new Recordshelf_Log_Writer_MongoDb(new Mongo($mongoDbServer),
$mongoDbName, $mongoDbCollection);
if ('production' === $this->getEnvironment()) {
$priority = constant($config->log->get('priority', Zend_Log::CRIT));
$filter = new Zend_Log_Filter_Priority($priority);
$logger->addFilter($filter);
}
$logger->addWriter($writer);
$this->_logger = $logger;
Zend_Registry::set('log', $logger);
}
}controllers/ExampleController.php
<?php
class ExampleController extends Zend_Controller_Action
{
private $_logger = null;
public function init()
{
$this->_logger = Zend_Registry::get('log');
}
public function fooAction()
{
$this->_logger->log('A debug log message from within action ' .
$this->getRequest()->getActionName(), Zend_Log::DEBUG);
}
public function barAction()
{
$this->_logger->log('A debug log message from within ' .
__METHOD__, Zend_Log::DEBUG);
}
}Accessing the log database with a Zend_Tool project provider
After handling the application-wide logging with the MongoDb writer sooner or later the issue to access the gathered log entries will rise. For this mundane and recurring use case the ProjectProvider provider of the Zend_Tool framework is an acceptable candidate to hook a custom action into the Zend_Tool environment of a given project. Therefor a new Zend_Tool_Project Project provider is first scaffolded via the forthcoming command.sudo zf create project-provider mongodb-logs filterSecond the generated provider skeleton its filter action is enliven with the logic to query the MongoDb database and the stored log collection. The action to come accepts three arguments to filter the stored log entry results by a specific date in the format of 'YYYY-MM-DD' and a given Zend_Log priority (currently limited to the constants defined in Zend_Log) in a specific application environment. The next listing shows the implementation of the import action of the MongodbLogsProvider project provider; which is clearly, as it's length indicates, in need for a clean-up task.
providers/Mongodb-logsProvider.php
<?php
require_once 'Zend/Tool/Project/Provider/Abstract.php';
require_once 'Zend/Tool/Project/Provider/Exception.php';
require_once 'Zend/Date.php';
require_once 'Zend/Validate/Date.php';
require_once 'Zend/Log.php';
require_once 'Zend/Config/Ini.php';
class MongodbLogsProvider extends Zend_Tool_Project_Provider_Abstract
{
public function filter($date = null, $logPriority = null,
$env = 'development')
{
$ref = new Zend_Reflection_Class('Zend_Log');
$logPriorities = $ref->getConstants();
if (in_array(strtoupper($date), array_keys($logPriorities)) ||
in_array(strtoupper($date), array_values($logPriorities))) {
$logPriority = $date;
$date = null;
}
if (!is_null($date)) {
$validator = new Zend_Validate_Date();
if (!$validator->isValid($date)) {
$exceptionMessage = "Given date '{$date}' is not a valid date.";
throw new Zend_Tool_Project_Provider_Exception($exceptionMessage);
}
$dateArray = array();
list($dateArray['year'], $dateArray['month'], $dateArray['day']) =
explode('-', $date);
$date = new Zend_Date($dateArray);
} else {
$date = new Zend_Date();
}
$date = $date->toString('Y-MM-dd');
if (!is_null($logPriority)) {
if (!is_numeric($logPriority)) {
$logPriority = strtoupper($logPriority);
if (!in_array($logPriority, array_keys($logPriorities))) {
$exceptionMessage = "Given priority '{$logPriority}' is not defined.";
throw new Zend_Tool_Project_Provider_Exception($exceptionMessage);
} else {
$logPriority = $logPriorities[$logPriority];
}
}
if (!in_array($logPriority, array_values($logPriorities))) {
$exceptionMessage = "Given priority '{$logPriority}' is not defined.";
throw new Zend_Tool_Project_Provider_Exception();
}
$priorities = array_flip($logPriorities);
$priorityName = $priorities[$logPriority];
}
if ($env !== 'development' && $env !== 'production') {
$exceptionMessage = "Unsupported environment '{$env}' provided.";
throw new Zend_Tool_Project_Provider_Exception();
}
$config = new Zend_Config_Ini('./application/configs/application.ini',
$env);
$applicationName = $config->app->get('name', 'recordshelf');
$mongoDbServer = $config->log->mongodb->get('server', '127.0.0.1');
$mongoDbName = $config->log->mongodb->get('db', "{$applicationName}_logs");
$mongoDbCollection = $config->log->mongodb->get('collection', 'entries');
try {
$connection = new Mongo($mongoDbServer);
$db = $connection->selectDB($mongoDbName)->createCollection(
$mongoDbCollection);
} catch (MongoConnectionException $e) {
throw new Zend_Tool_Project_Provider_Exception($e->getMessage());
}
$dateRegex = new MongoRegex("/$date.*/i");
if (is_null($logPriority)) {
$query = array('timestamp' => $dateRegex);
$appendContentForResults = "Found #amountOfEntries# log entrie(s) "
. "on {$date}";
$appendContentForNoResults = "Found no log entries on {$date}";
} else {
$query = array('priority' => (int) $logPriority,
'timestamp' => $dateRegex
);
$appendContentForResults = "Found #amountOfEntries# log entrie(s) "
. "for priority {$priorityName} on {$date}";
$appendContentForNoResults = "Found no log entries for priority "
. "{$priorityName} on {$date}";
}
$cursor = $db->find($query);
$amountOfEntries = $cursor->count();
if ($amountOfEntries > 0) {
$content = str_replace('#amountOfEntries#', $amountOfEntries,
$appendContentForResults);
$this->_registry->getResponse()->appendContent($content);
foreach ($cursor as $id => $value) {
$content = "{$id}: {$value['timestamp']} > ";
if (is_null($logPriority)) {
$content.= "[{$value['priorityName']}] ";
}
$content.= "{$value['message']}";
$this->_registry->getResponse()->appendContent($content);
}
} else {
$content = $appendContentForNoResults;
$this->_registry->getResponse()->appendContent($content);
}
$connection->close();
}
}The coming outro screenshots show two use cases for the filter action of the MongodbLogsProvider issued against the zf command line client. The first screenshot shows the use case where all log entries for the current day are queried, while the second one shows the use case where all log entries for a specific date and log priority are queried and fed back to the user.
Kicking off custom Phing task development with TextMate 22 Aug 2009 5:08 AM (15 years ago)
As a reader of this blog you migth have noticed that from time to time I like to utilize Phing's ability to write custom tasks. Though that's not an everyday routine for me and therefor I might, depending on my form of the day, end up with some real smelly code where for example the task's properties validation is handled in the task's main worker method. This is actually a bad habit/practice I'm aware of and to improve my future endeavours in custom Phing task development, I bended TextMate's snippet feature to my needs.
Snippets in TextMate are a very powerful feature that can be used to insert code that you do not want to type again and again, or like in my case might have forgotten over a certain time.
The next code listing shows the snippet providing a basic custom Phing task class skeleton which can be utilized over and over at the beginning of the implementation activities.
<?phpTo apply the snippet, after installing it, on a PHP source file it can either be selected from the Bundles menue or more comfortable via the assigned tab trigger i.e. ctask. After triggering the snippet it's possible to properly name the task under development and dynamically set it's first property, which is also treated as a mandatory property in the extracted _validateProperties method.
require_once 'phing/Task.php';
class ${1:CustomName}Task extends Task
{
private \$_${2:property} = null;
/**
* @param string \$${2:property} ${3:description}
*/
public function set${2/./\u$0/}(\$${2:property})
{
\$this->_${2:property} = trim(\$${2:property});
}
/**
* Initializes the task environment if necessary
*/
public function init()
{
}
/**
* Does the task main work or delegates it
* @throws BuildException
*/
public function main()
{
\$this->_validateProperties();
}
/**
* Validates the task properties
* @throws BuildException
*/
private function _validateProperties()
{
if (is_null(\$this->_${2:property})) {
throw new BuildException('${4:message}.');
}$0
}
}
The outro image shows the above stated snippet in the TextMate Bundle Editor and it's configuration.
%3Cbr%20/%3E%20%20%20%20%7B%3Cbr%20/%3E%20%20%20%20%20%20%20%20%5C$this-%3E_$%7B2:property%7D%20%3D%20trim(%5C$$%7B2:property%7D);%3Cbr%20/%3E%20%20%20%20%7D%3Cbr%20/%3E%20%20%20%20/**%3Cbr%20/%3E%20%20%20%20%20*%20Initializes%20the%20task%20environment%20if%20necessary%3Cbr%20/%3E%20%20%20%20%20*/%3Cbr%20/%3E%20%20%20%20public%20function%20init()%3Cbr%20/%3E%20%20%20%20%7B%3Cbr%20/%3E%20%20%20%20%7D%3Cbr%20/%3E%20%20%20%20/**%3Cbr%20/%3E%20%20%20%20%20*%20Does%20the%20task%20main%20work%20or%20delegates%20it%3Cbr%20/%3E%20%20%20%20%20*%20@throws%20BuildException%3Cbr%20/%3E%20%20%20%20%20*/%3Cbr%20/%3E%20%20%20%20public%20function%20main()%3Cbr%20/%3E%20%20%20%20%7B%3Cbr%20/%3E%20%20%20%20%20%20%20%20%5C$this-%3E_validateProperties();%3Cbr%20/%3E%20%20%20%20%7D%3Cbr%20/%3E%20%20%20%20/**%3Cbr%20/%3E%20%20%20%20%20*%20Validates%20the%20task%20properties%3Cbr%20/%3E%20%20%20%20%20*%20@throws%20BuildException%3Cbr%20/%3E%20%20%20%20%20*/%3Cbr%20/%3E%20%20%20%20private%20function%20_validateProperties()%3Cbr%20/%3E%20%20%20%20%7B%3Cbr%20/%3E%20%20%20%20%20%20%20%20if%20(is_null(%5C$this-%3E_$%7B2:property%7D))%20%7B%3Cbr%20/%3E%20%20%20%20%20%20%20%20%20%20%20%20throw%20new%20BuildException('$%7B4:message%7D.');%3Cbr%20/%3E%20%20%20%20%20%20%20%20%7D$0%3Cbr%20/%3E%20%20%20%20%7D%3Cbr%20/%3E%7D%3C/pre%3ETo%20apply%20the%20snippet,%20after%20installing%20it,%20on%20a%20PHP%20source%20file%20it%20can%20either%20be%20selected%20from%20the%20%3Ca%20href%3D%22http://manual.macromates.com/en/bundles%23bundles%22%20target%3D%22_self%22%3EBundles%3C/a%3E%20menue%20or%20more%20comfortable%20via%20the%20assigned%20tab%20trigger%20i.e.%20%3Cem%3Ectask%3C/em%3E.%20After%20triggering%20the%20snippet%20it's%20possible%20to%20properly%20name%20the%20task%20under%20development%20and%20dynamically%20set%20it's%20first%20property,%20which%20is%20also%20treated%20as%20a%20mandatory%20property%20in%20the%20extracted%20%3Cem%3E_validateProperties%3C/em%3E%20method.%3Cbr%20/%3E%3Cbr%20/%3EThe%20outro%20image%20shows%20the%20above%20stated%20snippet%20in%20the%20TextMate%20Bundle%20Editor%20and%20it's%20configuration.%3Cbr%20/%3E%3Cbr%20/%3E%3Ca%20href%3D%22http://www.flickr.com/photos/raphaelstolt/3844647803/%22%20title%3D%22Phing%20snippet%20in%20the%20TextMate%20Bundle%20Editor%20by%20Raphael%20Stolt,%20on%20Flickr%22%3E%3Cimg%20src%3D%22http://farm4.static.flickr.com/3550/3844647803_3175559fed_o.gif%22%20width%3D%22616%22%20height%3D%22431%22%20border%3D%220%22%20alt%3D%22Phing%20snippet%20in%20the%20TextMate%20Bundle%20Editor%22%20/%3E%3C/a%3E%3Cbr%20/%3E%3Cbr%20/%3E)
Scaffolding, implementing and using project specific Zend_Tool_Project_Providers 4 Jul 2009 6:41 AM (15 years ago)
Working on a project involving several legacy data migration tasks, I got curious what the Zend_Tool_Project component of the Zend Framework offers to create project specific providers for the above mentioned tasks or ones of similar nature. Therefore the following post will try to show how these providers can be developed in an iterative manner by scaffolding them via the capabilities of the Zend_Tool_Project ProjectProvider provider, enlived with action/task logic, and be used in the project scope.
Scaffolding project specific providers
All following steps assume there is a project available i.e. recordshelf initially created with the Zend_Tool_Project Project provider and that the forthcoming commands are issued from the project root directory against the zf command line client. The scaffolding of a project specific provider can be triggered via the create action of the ProjectProvider provider by passing in the name of the provider i.e. csv and it's intended actions. As the next console snippet shows it'spossible to specify several actions as a comma separated list.
sudo zf create project-provider csv importSpecials,importSummersaleAfter running the command the project's profile .zfproject.xml has been modified and a new providers directory exists in the project root directory containing the scaffolded Csv provider. The next code snippet shows the initial Csv provider class skeleton and its two empty action methods named importSpecials and importSummersale. At the point of this writing, using the Zend Framework 1.8.4 and PHP 5.2.10 on a Mac OS X system the generated Csv provider code or the mapping in the .zfproject.xml is incorrect, but can be fixed by renaming the class from CsvProvider to Csv.
<?php
require_once 'Zend/Tool/Project/Provider/Abstract.php';
require_once 'Zend/Tool/Project/Provider/Exception.php';
class CsvProviderextends Zend_Tool_Project_Provider_Abstract
{
public function importSpecials()
{
/** @todo Implementation */
}
public function importSummersale()
{
/** @todo Implementation */
}
}
Implementing the action logic
Having the project provider class skeleton ready to get going, it's time to enliven the actions with their intended features by using either other components of the Zend Framework, any suitable third party library or plain-vanilla PHP. For the sake of brevity I decided to implement only the importSpecials action which transforms the data of a known CSV file structure into a relevant database table. The CSV parsing steps shown next might not be that sophisticated, as their sole purpose is to illustrate an exemplary implementation of a project specific provider action.<?php
require_once 'Zend/Tool/Project/Provider/Abstract.php';
require_once 'Zend/Tool/Project/Provider/Exception.php';
class Csv extends Zend_Tool_Project_Provider_Abstract
{
private function _isProjectProviderSupportedInProject(Zend_Tool_Project_Profile $profile,
$projectProviderName)
{
$projectProviderResource = $this->_getProjectProfileResource($profile,
$projectProviderName);
return $projectProviderResource instanceof Zend_Tool_Project_Profile_Resource;
}
private function _isActionSupportedByProjectProvider(Zend_Tool_Project_Profile $profile,
$projectProviderName, $actionName)
{
$projectProviderResource = $this->_getProjectProfileResource($profile,
$projectProviderName);
$projectProviderAttributes = $projectProviderResource->getContext()
->getPersistentAttributes();
return in_array($actionName, explode(',', $projectProviderAttributes['actionNames']));
}
private function _getProjectProfileResource(Zend_Tool_Project_Profile $profile,
$projectProviderName)
{
$profileSearchParams[] = 'ProjectProvidersDirectory';
$profileSearchParams['ProjectProviderFile'] =
array('projectProviderName' => strtolower($projectProviderName));
return $profile->search($profileSearchParams);
}
public function importSpecials($csvFile, $env = 'development')
{
$relatedTablename = 'specials';
if (!$this->_isProjectProviderSupportedInProject($profile, __CLASS__)) {
throw new Exception("ProjectProvider Csv is not supported in this project.");
}
if (!$this->_isActionSupportedByProjectProvider($profile, __CLASS__, __FUNCTION__)) {
$exceptionMessage = "Action 'importSpecials' is not supported by "
. "the Csv ProjectProvider in this project.";
throw new Exception($exceptionMessage);
}
if (!file_exists($csvFile)) {
throw new Exception("Given csv-file '{$csvFile}' doesn't exist.");
}
$importEnvironment = trim($env);
if ($importEnvironment !== 'development' && $importEnvironment !== 'production') {
throw new Exception("Unsupported environment '{$importEnvironment}' provided.");
}
$csvHandle = fopen($csvFile, "r");
if (!$csvHandle) {
throw new Exception("Unable to open given csv-file '{$csvFile}'.");
}
$config = new Zend_Config_Ini('./application/configs/application.ini',
$importEnvironment);
$db = Zend_Db::factory($config->database);
$db->query("TRUNCATE TABLE {$relatedTablename}");
echo "Truncated the project '{$relatedTablename}' database table." . PHP_EOL;
$rowCount = $insertCount = 0;
while (($csvLine = fgetcsv($csvHandle)) !== false) {
if ($rowCount > 0) {
$insertRow = array(
'product_name' => $csvLine[0],
'product_image_path' => $csvLine[1],
'price' => $csvLine[2],
'special_until' => $csvLine[3]
);
$db->insert($relatedTablename, $insertRow);
++$insertCount;
}
++$rowCount;
}
fclose($csvHandle);
$importMessage = "Imported {$insertCount} rows into the project "
. "'{$relatedTablename}' database table.";
echo $importMessage;
}
...
}
Making providers and actions pretendable
To make project specific providers its actions pretendable and thereby providing some kind of user documentation the provider classes have to implement a marker interface called Zend_Tool_Framework_Provider_Pretendable. For making a action of a provider pretendable and giving some feedback to the user, the request is checked if the action has been issued in the pretend mode; which is possible by adding -p option to the issued zf command line client command. The next code snippet shows how the above stated Csv provider and its importSpecials action is made pretendable.<?php
require_once 'Zend/Tool/Project/Provider/Abstract.php';
require_once 'Zend/Tool/Project/Provider/Exception.php';
class Csv extends Zend_Tool_Project_Provider_Abstract implements
Zend_Tool_Framework_Provider_Pretendable
{
public function importSpecials($csvFile, $env = 'development')
{
...
if ($this->_registry->getRequest()->isPretend()) {
$pretendMessage = "I would import the specials data provided in {$csvFile} "
. "into the project '{$relatedTablename}' database table.";
echo $pretendMessage;
} else {
...
}
}
...
}
Using project specific providers
To use the bundled up capabilities of project specific providers, these have to made accessable to the zf command line client by putting them in the include_path. Currently I discovered no best practice for doing so only for single project scopes and simply added the path to the project to my php.ini and thereby global include_path; another approach might be to add the project name as a prefix to the Provider. After doing so it's possible to get an overview of all with the Zend_Tool_Project shipped providers plus the project specific providers and their offered actions by issuing the zf --help command as shown in the next screenshot. To ensure that project specific providers and its actions are only runnable in projects which support them, it is necessary to check if these and the offered action exists as resources in the project its profile .zfproject.xml file as shown in the implementation of the importSpecials action in one of above code snippets.
As shown in the previous screenshot the first character of the project specific providers are omitted, this is another minor bug which might be fixed in one of the forthcoming Zend Framework releases. The current workaround for this issue is simply to type the command exactly as shown in the help. The outro screenshot shows how the import-specials action of the project specific Csv provider is issued against the zf command line client and its provided user feedback after an successfull import against the projects development database.
Testing Phing buildfiles with PHPUnit 10 May 2009 6:15 AM (15 years ago)
While transforming some of the Ant buildfile refactorings described in Julian Simpson's seminal essay into a Phing context, it felt plainly wrong that I didn't have any tests for the buildfile to back me up on obtaining the pristine behaviour throughout the process. While Ant users can rely on an Apache project called AntUnit there are currently no tailor-made tools available for testing or verifying Phing buildfiles. Therefor I took a weekend off, locked myself in the stuffy lab, and explored the abilities to test Phing buildfiles respectively their included properties, targets and tasks with the PHPUnit testing framework. In case you'd like to take a peek at the emerged lab jottings, keep on scanning.
Introducing the buildfile under test
The buildfile that will be used as an example is kept simple, and contains several targets ranging from common ones like initializing the build environment by creating the necessary directories to more specific ones like pulling an external artifact from GitHub. To get an overview of the buildfile under test have a look at the following listing.<?xml version="1.0" encoding="UTF-8"?>
<project name="test-example" default="build" basedir=".">
<property name="project.basedir" value="." override="true" />
<property name="github.repos.dir" value="${project.basedir}/build/github-repos" override="true" />
<target name="clean" depends="clean-github-repos" description="Removes runtime build artifacts">
<delete dir="${project.basedir}/build" includeemptydirs="true" verbose="false" failonerror="true" />
<delete dir="${project.basedir}/build/reports" includeemptydirs="true" verbose="false" failonerror="true" />
</target>
<target name="clean-github-repos" description="Removes runtime build artifacts">
<delete dir="${github.repos.dir}" includeemptydirs="true" failonerror="true" />
</target>
<target name="-log-build" description="A private target which should only be invoked internally">
<!-- omitted -->
</target>
<target name="build" depends="clean" description="Builds the distributable product">
<!-- omitted -->
</target>
<target name="database-setup" description="Sets up the database structure">
<!-- omitted -->
</target>
<target name="init" description="Initalizes the build by creating directories etc">
<mkdir dir="${project.basedir}/build/logs/performance/" />
<mkdir dir="${project.basedir}/build/doc" />
<mkdir dir="${project.basedir}/build/reports/phploc" />
</target>
<target name="init-ad-hoc-tasks"
description="Initalizes the ad hoc tasks for reusability in multiple targets">
<adhoc-task name="github-clone"><![CDATA[
class Github_Clone extends Task {
private $repository = null;
private $destDirectory = null;
function setRepos($repository) {
$this->repository = $repository;
}
function setDest($destDirectory) {
$this->destDirectory = $destDirectory;
}
function main() {
// Get project name from repos Uri
$projectName = str_replace('.git', '',
substr(strrchr($this->repository, '/'), 1));
$gitCommand = 'git clone ' . $this->repository . ' ' .
$this->destDirectory . '/' . $projectName;
exec(escapeshellcmd($gitCommand), $output, $return);
if ($return !== 0) {
throw new BuildException('Git clone failed');
}
$logMessage = 'Cloned Git repository ' . $this->repository .
' into ' . $this->destDirectory . '/' . $projectName;
$this->log($logMessage);
}
}
]]></adhoc-task>
<echo message="Intialized github-clone ad hoc task." />
</target>
<target name="github" depends="init-ad-hoc-tasks, clean-github-repos"
description="Clones given repositories from GitHub">
<github-clone repos="git://github.com/raphaelstolt/phploc-phing.git" dest="${github.repos.dir}" />
</target>
</project>Testing the buildfile
All tests for the buildfile under test will be bundled, like 'normal' tests, in a class i.e. BuildfileTest extending the PHPUnit_Framework_TestCase class. When testing buildfiles it's possible to build some tests around the actual buildfile XML structure, by utilizing the xpath method of PHP's SimpleXMLElement class and asserting against the XPath query results, or around the dispatching of specific targets and asserting against the expected build artifacts. Furthermore these two identified groups, structure and artifact, can be used to organize the accumulating tests via PHPUnit's @group annotation.To be able to dispatch specific build targets and feed them with properties if necessary I additionally developed a very basic build runner shown in the next code listing.
<?php
class Phing_Buildfile_Runner {
private $_buildfilePath = null;
public function __construct($buildfilePath) {
if (!file_exists($buildfilePath)) {
throw new Exception("Buildfile '{$buildfilePath}' doesn't exist");
}
$this->buildfilePath = realpath($buildfilePath);
}
public function runTarget($targets = array(), $properties = array()) {
$runTargetCommand = "phing " . "-f {$this->buildfilePath} ";
if (count($targets) > 0) {
foreach ($targets as $target) {
$runTargetCommand.= $target . " ";
}
}
if (count($properties) > 0) {
foreach ($properties as $property => $value) {
$runTargetCommand.= "-D{$property}={$value} ";
}
}
exec(escapeshellcmd($runTargetCommand), $output, $return);
return array('output' => $output, 'return' => $return);
}
}Out of the box PHPUnit's assertion pool provides all the utilities to test buildfiles; although it would be cleaner to create domain specfic assertions for this testing domain this technique will be ignored for the sake of brevity.After an initial 1000ft view on how to test buildfiles let's jump into the actual testing of a structural aspect of the buildfile under test. The test to come shows how to verify that a clean target is defined for playing along in the build orchestra by querying a XPath expression against the buildfile XML and asserting that a result is available.
/**
* @test
* @group structure
*/
public function buildfileShouldContainACleanTarget() {
$xml = new SimpleXMLElement($this->_buildfileXml);
$cleanElement = $xml->xpath("//target[@name='clean']");
$this->assertTrue(count($cleanElement) > 0, "Buildfile doesn't contain a clean target");
}The next artifactual test raises the bar an inch, by verifying that the defined init target of the build does initialize the build environment correctly, or to pick up the orchestra metaphor again that the specific instrument plays along and holds the directed tone. Therefor the build runner executes the target and afterwards asserts a list of expected artifacts against the current state of the build process./**
* @test
* @group artifact
*/
public function initTargetShouldCreateInitialBuildArtifacts() {
$this->_isTearDownNecessary = true;
$this->_buildfileRunner->runTarget(array('init'));
$expectedInitArtifacts = array(
"{$this->_buildfileBasedir}/build",
"{$this->_buildfileBasedir}/build/logs/performance/",
"{$this->_buildfileBasedir}/build/doc",
"{$this->_buildfileBasedir}/build/reports"
);
foreach ($expectedInitArtifacts as $artifact) {
$this->assertFileExists($artifact, "Expected file '{$artifact}' doesn't exist");
}
}The next code listing shows the whole picture of the BuildfileTest class containing additional test methods verifying different aspects of the buildfile under test and also the innards of the setup and teardown method.The main assignment of the setup method is to load the XML of the buildfile under test and to intialize the build runner so an instance is available for an use in artifactual tests. The teardown method its sole responsibility is to reset the build state by running the clean target of the buildfile.
<?php
require_once 'PHPUnit/Framework.php';
require_once 'Phing/Buildfile/Runner.php';
class ExampleBuildfileTest extends PHPUnit_Framework_TestCase {
protected $_buildfileXml = null;
protected $_buildfileName = null;
protected $_buildfileBasedir = null;
protected $_buildfileRunner = null;
protected $_isTearDownNecessary = false;
protected function setUp() {
$this->_buildfileName = realpath('../../build.xml');
$this->_buildfileBasedir = dirname($this->_buildfileName);
$this->_buildfileXml = file_get_contents($this->_buildfileName);
$this->_buildfileRunner = new Phing_Buildfile_Runner(
$this->_buildfileName);
}
protected function tearDown() {
if ($this->_isTearDownNecessary) {
$this->_buildfileRunner->runTarget(array('clean'));
}
}
/**
* @test
* @group structure
*/
public function targetBuildShouldBeTheDefaultTarget() {
$xml = new SimpleXMLElement($this->_buildfileXml);
$xpath = "//@default";
$defaultElement = $xml->xpath($xpath);
$this->assertSame('build', trim($defaultElement[0]->default),
"Buildfile doesn't have a default target named 'build'"
);
}
/**
* @test
* @group structure
*/
public function propertyGithubReposDirShouldBeSet() {
$xml = new SimpleXMLElement($this->_buildfileXml);
$xpath = "//property[@name='github.repos.dir']/@value";
$valueElement = $xml->xpath($xpath);
$this->assertTrue($valueElement[0] instanceof SimpleXMLElement,
"Buildfile doesn't contain a 'github.repos.dir' property"
);
$this->assertGreaterThan(1, strlen($valueElement[0]->value));
}
/**
* @test
* @group structure
*/
public function buildfileShouldContainACleanTarget() {
$xml = new SimpleXMLElement($this->_buildfileXml);
$cleanElement = $xml->xpath("//target[@name='clean']");
$this->assertTrue(count($cleanElement) > 0,
"Buildfile doesn't contain a clean target"
);
}
/**
* @test
* @group structure
*/
public function targetLogBuildShouldBeAPrivateOne() {
$xml = new SimpleXMLElement($this->_buildfileXml);
$nameElement = $xml->xpath("//target[@name='-log-build']");
$this->assertTrue(count($nameElement) > 0,
'Log build target is not a private target'
);
}
/**
* @test
* @group structure
*/
public function targetBuildShouldDependOnCleanTarget() {
$xml = new SimpleXMLElement($this->_buildfileXml);
$xpath = "//target[@name='build']/@depends";
$dependElement = $xml->xpath($xpath);
$this->assertTrue(count($dependElement) > 0,
'Target build contains no depends attribute'
);
$dependantTasks = array_filter(explode(' ',
trim($dependElement[0]->depends))
);
$this->assertContains('clean', $dependantTasks, "Target build doesn't
depend on the clean target"
);
}
/**
* @test
* @group structure
*/
public function allDefinedTargetsShouldHaveADescriptionAttribute() {
$xml = new SimpleXMLElement($this->_buildfileXml);
$xpath = "//target";
$targetElements = $xml->xpath($xpath);
$describedTargetElements = array();
foreach ($targetElements as $index => $targetElement) {
$targetDescription = trim($targetElement->attributes()->description);
if ($targetDescription !== '') {
$describedTargetElements[] = $targetDescription;
}
}
$this->assertEquals(count($targetElements),
count($describedTargetElements),
'Description not for all targets set'
);
}
/**
* @test
* @group structure
*/
public function githubCloneAdhocTaskShouldBeDefined() {
$xml = new SimpleXMLElement($this->_buildfileXml);
$xpath = "//target[@name='init-ad-hoc-tasks']/adhoc-task";
$adhocElement = $xml->xpath($xpath);
$this->assertSame('github-clone',
trim($adhocElement[0]->attributes()->name),
"Ad hoc task 'github-clone' isn't defined"
);
}
/**
* @test
* @group artifact
*/
public function initTargetShouldCreateInitialBuildArtifacts() {
$this->_isTearDownNecessary = true;
$this->_buildfileRunner->runTarget(array('init'));
$expectedInitArtifacts = array(
"{$this->_buildfileBasedir}/build",
"{$this->_buildfileBasedir}/build/logs/performance/",
"{$this->_buildfileBasedir}/build/doc",
"{$this->_buildfileBasedir}/build/reports"
);
foreach ($expectedInitArtifacts as $artifact) {
$this->assertFileExists($artifact,
"Expected file '{$artifact}' doesn't exist"
);
}
}
/**
* @test
* @group artifact
*/
public function sqlFilesForDatabaseSetupTargetShouldBeAvailable() {
$expectedSqlFiles = array(
"{$this->_buildfileBasedir}/sqlfiles",
"{$this->_buildfileBasedir}/sqlfiles/session-storage.sql",
"{$this->_buildfileBasedir}/sqlfiles/acl.sql",
"{$this->_buildfileBasedir}/sqlfiles/log.sql"
);
foreach ($expectedSqlFiles as $sqlFile) {
$this->assertFileExists($sqlFile,
"SQL file '{$sqlFile}' doesn't exist"
);
}
}
/**
* @test
* @group artifact
*/
public function githubTargetShouldFetchExpectedRepository() {
$this->_isTearDownNecessary = true;
$this->_buildfileRunner->runTarget(array('github'));
$expectedGitRepository = "{$this->_buildfileBasedir}/build/"
. "github-repos/phploc-phing/.git";
$this->assertFileExists($expectedGitRepository,
"Github target doesn't fetch the expected 'phploc-phing' repository"
);
}
}The outro screenshot shows the above stated test class run against the example buildfile on a Mac OS X system utilizing the --colors option; which by the way comes in really handy in combination with Stakeout.rb during the process of refactoring or extending/creating buildfiles the test-driven way.
Creating and using Phing ad hoc tasks 17 Apr 2009 4:48 PM (16 years ago)
Sometimes there are build scenarios where you'll badly need a functionality, like adding a MD5 checksum file to a given project, that isn't provided neither by the available Phing core nor the optional tasks. Phing supports developers with two ways for extending the useable task pool: by writing 'outline' tasks that will end up in a directory of the Phing installation or by utilizing the AdhocTaskdefTask, which allows to define custom tasks in the buildfile itself. The following post will try to outline how to define and use these inline tasks, by sketching an ad hoc task that enables the build orchestra to clone Git repositories from GitHub during a hypothetical workbench setup.
Creating the inline/ad hoc task
The AdhocTaskdefTask expects a name attribute i.e. github-clone for the XML element which will later referr to the ad hoc task and a CDATA section hosting the task implementation. Similar to 'outline' tasks the ad hoc task extends Phing's Task class, configures the task via attributes and holds the logic to perform. Unfortunately inline task implementations don't allow to require or include external classes available in the include_path, like Zend_Http_Client which I initially tried to use for an example task fetching short Urls from is.gd. This limits the available functions and classes to craft the task from to the ones built into PHP. The following buildfile snippet shows the implementation of the github-clone ad hoc task which is wrapped by a private target to encourage reusability and limit it's callability.<target name="-init-ad-hoc-tasks"
description="Initializes the ad hoc task(s)">
<adhoc-task name="github-clone"><![CDATA[
class Github_Clone extends Task {
private $repository = null;
private $destDirectory = null;
function setRepos($repository) {
$this->repository = $repository;
}
function setDest($destDirectory) {
$this->destDirectory = $destDirectory;
}
function main() {
// Get project name from repos Uri
$projectName = str_replace('.git', '',
substr(strrchr($this->repository, '/'), 1));
$gitCommand = 'git clone ' . $this->repository . ' ' .
$this->destDirectory . '/' . $projectName;
exec(escapeshellcmd($gitCommand), $output, $return);
if ($return !== 0) {
throw new BuildException('Git clone failed');
}
$logMessage = 'Cloned Git repository ' . $this->repository .
' into ' . $this->destDirectory . '/' . $projectName;
$this->log($logMessage);
}
}
]]></adhoc-task>
<echo message="Initialized github-clone ad hoc task." />
</target>
Using the ad hoc task
With the ad hoc task in the place to be, it's provided functionality can now be used from any target using the tasks XML element according to the given name i.e. github-clone in the AdhocTaskdefTask element earlier and by feeding it with the required attributes i.e. repos and dest. The next snippet allows you to take a peek at the complete buildfile with the ad hoc task in action.<?xml version="1.0" encoding="UTF-8"?>
<project name="recordshelf" default="init-work-bench" basedir=".">
<property name="github.repos.dir" value="./github-repos" override="true" />
<target name="init-work-bench"
depends="-init-ad-hoc-tasks, -clone-git-repos"
description="Initializes the hypothetical workbench">
<echo message="Initialized workbench." />
</target>
<target name="-clean-git-repos"
description="Removes old repositories before initializing a new workbench">
<delete dir="${github.repos.dir}" includeemptydirs="true" failonerror="true" />
</target>
<target name="-init-ad-hoc-tasks"
description="Initializes the ad hoc task(s)">
<adhoc-task name="github-clone"><![CDATA[
class Github_Clone extends Task {
private $repository = null;
private $destDirectory = null;
function setRepos($repository) {
$this->repository = $repository;
}
function setDest($destDirectory) {
$this->destDirectory = $destDirectory;
}
function main() {
// Get project name from repos Uri
$projectName = str_replace('.git', '',
substr(strrchr($this->repository, '/'), 1));
$gitCommand = 'git clone ' . $this->repository . ' ' .
$this->destDirectory . '/' . $projectName;
exec(escapeshellcmd($gitCommand), $output, $return);
if ($return !== 0) {
throw new BuildException('Git clone failed');
}
$logMessage = 'Cloned Git repository ' . $this->repository .
' into ' . $this->destDirectory . '/' . $projectName;
$this->log($logMessage);
}
}
]]></adhoc-task>
<echo message="Initialized github-clone ad hoc task." />
</target>
<target name="-clone-git-repos" depends="-clean-git-repos"
description="Clones the needed Git repositories from GitHub">
<github-clone repos="git://github.com/abc/abc.git"
dest="${github.repos.dir}" />
<github-clone repos="git://github.com/xyz/xyz.git"
dest="${github.repos.dir}" />
</target>
</project>
Favouring inline over 'outline' tasks?
The one big advantage of using inline tasks over 'outline' tasks is that they are distributed with the buildfile and are instantly available without the need to modify the Phing installation. Some severe disadvantages of inline tasks are the limitation to use only the core PHP functions and classes for the implementation, the introduction of an additional hurdle to verify the task behaviour via PHPUnit as it's located in a CDATA section of the buildfile and the fact that the use of several inline tasks will blow up the buildfile, and thereby obfuscate the build flow.Regrettably Phing doesn't provide an import task like Ant which might enable a refactoring to pull the ad hoc task definitions into a seperate XML file and include them at buildtime; in case you might have some expertise or ideas for a suitable workaround hit me with a comment. So far I tried to get it working, with no success, by utilizing Phing's PhingTask and XML's external entities declaration.
Using Haml & Sass from a Rake task 31 Mar 2009 6:02 AM (16 years ago)
Some time ago I had the 'lightning' idea to implement another Rake automation to support my current blogging workflow, which at the moment consists of finding a sparkling idea to blog about, write it out in WriteRoom and refine the post in TextMate before publishing. As this process was a recurring and copy & paste driven event, I strove for an automation supporting this workflow. So unsurprisingly the post will show my current solution to achieve this goal by utilizing Rake, Haml and Sass.
So what's that Haml and Sass thingy?
Haml (HTML Abstraction Markup Language) is a templating language/engine with the primary goal to make Markup DRY, beautiful and readable again. It has a very shallow learning curve and therefor is perfectly suited for programmers and designers alike. Haml is primarily targeted at making the views of Ruby on Rails, Merb or Sinatra web applications leaner, but as you will see later the Ruby implementation also can be used framework independently.Sass (Syntactically Awesome StyleSheets) is a module which comes bundled with Haml providing a meta-language/abstraction on top of CSS sharing the same goals and advantages as Haml.
Gluing Haml and Sass into a Rake task
To get going you first have to install Haml and Sass by running the gem command shown next.sudo gem install hamlWith Haml and Sass available it's about time to identify and outline the parts you want to automate, in my case it's the creation of a WriteRoom and/or a XHTML draft document for initial editings. So the parameters to pass into the task to come are the targeted editor(s), the title of the blog post to draft and a list of associated and whitespace separated category tags.
The XHTML document skeleton content and it's inline CSS are defined each in a separate Haml and Sass template file and will be rendered into the outcoming document along with the content passed into the Rake task. While the document skeleton for the WriteRoom draft document, due to it's brevity, is defined inside of the task itself. The following snippets are showing the mentioned Haml and Sass templates for the XHTML draft output file, which are located in the same directory as the Rake file.
Haml
!!! 1.1
%html
%head
%title= "#{title} - Draft"
%style{ :type => 'text/css' }= inline_css
%body
%h3= title
%h4.custom sub headline
%pre.consoleOutput console command
%pre.codeSnippet code snippet
%br/
= "Tags: #{tags.join ', '}"
Sass
bodyTo inject the dynamic content into the Haml template and have it rendered into the outcoming document, the values i.e. draft_title, draft_tags and draft_inline_css have to be made available to the template engine by passing them in a bundling Hash into the to_html alias method of the Haml Engine object like shown in the next Rake task.
:margin 5
:line-height 1.5em
:font small Trebuchet MS, Verdana, Arial, Sans-serif
:color #000000
h4
:margin-bottom 0.3em
.consoleOutput
:padding 6px
:background-color #000
:color rgb(20, 218, 62)
:font-size 12px
:font-weight bolder
.codeSnippet
:padding 3px
:background-color rgb(243, 243, 243)
:color rgb(93, 91, 91)
:font-size small
:border 1px solid #6A6565
task :default do
Rake::Task['blog_utils:create_draft_doc'].invoke
end
namespace :blog_utils do
desc 'Create a new draft document for a given title, category tags and editor'
task :create_draft_doc, [:title, :tags, :editor] do |t, args|
draft_title = args.title
draft_tags = args.tags.split(' ')
draft_target_editor = args.editor
raise_message = 'No title for draft provided'
raise raise_message if draft_title.nil?
raise_message = 'No tags for draft provided'
raise raise_message if draft_tags.nil?
draft_target_editor = '*' if draft_target_editor.nil?
raise_message = 'Unsupported target editor provided'
raise raise_message unless draft_target_editor == 'Textmate' ||
draft_target_editor == 'Writeroom' || draft_target_editor == '*'
if draft_target_editor == 'Writeroom' || draft_target_editor == '*'
draft_output_file = draft_title.gsub(' ', '_') + '.txt'
File.open(draft_output_file, 'w') do |draft_file_txt|
draft_file_txt.puts draft_title
draft_file_txt.puts
draft_file_txt.puts "Tags: #{draft_tags.join ', '}"
end
end
if draft_target_editor == 'Textmate' || draft_target_editor == '*'
template_sass_content, template_haml_content = ''
['haml', 'sass'].each do |template_type|
template = File.dirname(__FILE__) + "/draft_template.#{template_type}"
raise_message = "#{template_type.capitalize} template '#{template}' not found"
raise raise_message if !File.exists?(template)
template_sass_content = File.read(template) if template_type === 'sass'
template_haml_content = File.read(template) if template_type === 'haml'
end
require 'sass'
require 'haml'
draft_inline_css = Sass::Engine.new(template_sass_content).to_css
draft_document_content = Haml::Engine.new(template_haml_content).to_html(
Object.new, { :title => draft_title , :tags => draft_tags ,
:inline_css => draft_inline_css } )
draft_output_file = draft_title.gsub(' ', '_') + '.html'
File.open(draft_output_file, 'w') do |draft_file_html|
draft_file_html.puts(draft_document_content)
end
end
end
end
Easing invocation pain with alias
Now as the Rake task is implemented and waiting for demands it can be invoked by calling the task as shown in the next console snippet.sudo rake -f $HOME/Automations/Rakefile.rb blog_utils:create_draft_doc['Title','Tag1 TagN','Editor']As I'm not even close to being a console ninja and probably will have forgotten the task call structure before initiating the next blog post, I decided to add an easing and more memorizable alias to $HOME/.profile as shown next.
alias createdraft='sudo rake -f $HOME/Automations/Rakefile.rb blog_utils:create_draft_doc[$title,$tags,$editor]'The created alias now allows to invoke the Rake task in a nice and easy way as shown in the next console command.
createdraft title='Using Haml & Sass from a Rake task' tags='Rake Ruby' editor='Textmate'
Taking a peek at the generated draft document
After running the described Rake task I end up with the XHTML document shown in the outro code snippet, which then can be used for the further editing process. Of course I could have setup a TextMate Snippet to get me going, but that way I would have missed the opportunity to mess around with another amazing Ruby tool.<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html>
<head>
<title>Using Haml & Sass from a Rake task - Draft</title>
<style type='text/css'>
body {
margin: 5;
line-height: 1.5em;
font: small Trebuchet MS, Verdana, Arial, Sans-serif;
color: #000000; }
h4 {
margin-bottom: 0.3em; }
.consoleOutput {
padding: 6px;
background-color: #000;
color: rgb(20, 218, 62);
font-size: 12px;
font-weight: bolder; }
.codeSnippet {
padding: 3px;
background-color: rgb(243, 243, 243);
color: rgb(93, 91, 91);
font-size: small;
border: 1px solid #6A6565; }
</style>
</head>
<body>
<h3>Using Haml & Sass from a Rake task</h3>
<h4>sub headline</h4>
<pre class='consoleOutput'>console command</pre>
<pre class='codeSnippet'>code snippet</pre>
<br />
Tags: Rake, Ruby
</body>
</html>
Phplocing your projects with Phing 22 Feb 2009 4:35 AM (16 years ago)
When I started to play around with Ruby on Rails, my attention got somehow soon drawn to it's Rake stats task, which provides developers or more likely project managers with an overview of the actual project size. Exactly one month ago Sebastian Bergmann, of PHPUnit fame, started to implement a similar tool dubbed phploc which can give you an overview of the size for any given PHP project. As I wanted to automate the invocation of this handy tool and collect it's report output out of a Phing buildfile, I invested some time to develop a custom Phing task doing so. Thereby the following post will show you a possible implementation of this task and it's use in a buildfile.
Installing phploc
To setup phploc on your system simply install the phploc PEAR package available from the pear.phpunit.de channel as shown in the next commands. In case you already have installed PHPUnit via PEAR you can omit the channel-discover command.sudo pear channel-discover pear.phpunit.de
sudo pear install phpunit/phploc
Implementing the phploc task
As I already blogged about developing custom Phing task I'm only going to show the actual implementation and not dive into any details; alternatively you can also grab it from this public GitHub repository.<?php
require_once 'phing/Task.php';
require_once 'phing/BuildException.php';
require_once 'PHPLOC/Analyser.php';
require_once 'PHPLOC/Util/FilterIterator.php';
require_once 'PHPLOC/TextUI/ResultPrinter.php';
class PHPLocTask extends Task
{
protected $suffixesToCheck = null;
protected $acceptedReportTypes = null;
protected $reportDirectory = null;
protected $reportType = null;
protected $fileToCheck = null;
protected $filesToCheck = null;
protected $reportFileName = null;
protected $fileSets = null;
public function init() {
$this->suffixesToCheck = array('php');
$this->acceptedReportTypes = array('cli', 'txt', 'xml');
$this->reportType = 'cli';
$this->reportFileName = 'phploc-report';
$this->fileSets = array();
$this->filesToCheck = array();
}
public function setSuffixes($suffixListOrSingleSuffix) {
if (stripos($suffixListOrSingleSuffix, ',')) {
$suffixes = explode(',', $suffixListOrSingleSuffix);
$this->suffixesToCheck = array_map('trim', $suffixes);
} else {
array_push($this->suffixesToCheck, trim($suffixListOrSingleSuffix));
}
}
public function setFile(PhingFile $file) {
$this->fileToCheck = trim($file);
}
public function createFileSet() {
$num = array_push($this->fileSets, new FileSet());
return $this->fileSets[$num - 1];
}
public function setReportType($type) {
$this->reportType = trim($type);
}
public function setReportName($name) {
$this->reportFileName = trim($name);
}
public function setReportDirectory($directory) {
$this->reportDirectory = trim($directory);
}
public function main() {
if (!isset($this->fileToCheck) && count($this->fileSets) === 0) {
$exceptionMessage = "Missing either a nested fileset or the "
. "attribute 'file' set.";
throw new BuildException($exceptionMessage);
}
if (count($this->suffixesToCheck) === 0) {
throw new BuildException("No file suffix defined.");
}
if (is_null($this->reportType)) {
throw new BuildException("No report type defined.");
}
if (!is_null($this->reportType) &&
!in_array($this->reportType, $this->acceptedReportTypes)) {
throw new BuildException("Unaccepted report type defined.");
}
if (!is_null($this->fileToCheck) && !file_exists($this->fileToCheck)) {
throw new BuildException("File to check doesn't exist.");
}
if ($this->reportType !== 'cli' && is_null($this->reportDirectory)) {
throw new BuildException("No report output directory defined.");
}
if (count($this->fileSets) > 0 && !is_null($this->fileToCheck)) {
$exceptionMessage = "Either use a nested fileset or 'file' "
. "attribute; not both.";
throw new BuildException($exceptionMessage);
}
if (!is_null($this->reportDirectory) && !is_dir($this->reportDirectory)) {
$reportOutputDir = new PhingFile($this->reportDirectory);
$logMessage = "Report output directory does't exist, creating: "
. $reportOutputDir->getAbsolutePath() . '.';
$this->log($logMessage);
$reportOutputDir->mkdirs();
}
if ($this->reportType !== 'cli') {
$this->reportFileName.= '.' . trim($this->reportType);
}
if (count($this->fileSets) > 0) {
$project = $this->getProject();
foreach ($this->fileSets as $fileSet) {
$directoryScanner = $fileSet->getDirectoryScanner($project);
$files = $directoryScanner->getIncludedFiles();
$directory = $fileSet->getDir($this->project)->getPath();
foreach ($files as $file) {
if ($this->isFileSuffixSet($file)) {
$this->filesToCheck[] = $directory . DIRECTORY_SEPARATOR
. $file;
}
}
}
$this->filesToCheck = array_unique($this->filesToCheck);
}
if (!is_null($this->fileToCheck)) {
if (!$this->isFileSuffixSet($file)) {
$exceptionMessage = "Suffix of file to check is not defined in"
. " 'suffixes' attribute.";
throw new BuildException($exceptionMessage);
}
}
$this->runPhpLocCheck();
}
protected function isFileSuffixSet($filename) {
$pathinfo = pathinfo($filename);
$fileSuffix = $pathinfo['extension'];
return in_array($fileSuffix, $this->suffixesToCheck);
}
protected function runPhpLocCheck() {
$files = $this->getFilesToCheck();
$result = $this->getCountForFiles($files);
if ($this->reportType === 'cli' || $this->reportType === 'txt') {
$printer = new PHPLOC_TextUI_ResultPrinter;
if ($this->reportType === 'txt') {
ob_start();
$printer->printResult($result);
file_put_contents($this->reportDirectory
. DIRECTORY_SEPARATOR . $this->reportFileName,
ob_get_contents());
ob_end_clean();
$reportDir = new PhingFile($this->reportDirectory);
$logMessage = "Writing report to: "
. $reportDir->getAbsolutePath() . DIRECTORY_SEPARATOR
. $this->reportFileName;
$this->log($logMessage);
} else {
$printer->printResult($result);
}
} elseif ($this->reportType === 'xml') {
$xml = $this->getResultAsXml($result);
$reportDir = new PhingFile($this->reportDirectory);
$logMessage = "Writing report to: " . $reportDir->getAbsolutePath()
. DIRECTORY_SEPARATOR . $this->reportFileName;
$this->log($logMessage);
file_put_contents($this->reportDirectory . DIRECTORY_SEPARATOR
. $this->reportFileName, $xml);
}
}
protected function getFilesToCheck() {
if (count($this->filesToCheck) > 0) {
$files = array();
foreach ($this->filesToCheck as $file) {
$files[] = new SPLFileInfo($file);
}
} elseif (!is_null($this->fileToCheck)) {
$files = array(new SPLFileInfo($this->fileToCheck));
}
return $files;
}
protected function getCountForFiles($files) {
$count = array('files' => 0, 'loc' => 0, 'cloc' => 0, 'ncloc' => 0,
'eloc' => 0, 'interfaces' => 0, 'classes' => 0, 'functions' => 0);
$directories = array();
foreach ($files as $file) {
$directory = $file->getPath();
if (!isset($directories[$directory])) {
$directories[$directory] = TRUE;
}
PHPLOC_Analyser::countFile($file->getPathName(), $count);
}
if (!function_exists('parsekit_compile_file')) {
unset($count['eloc']);
}
$count['directories'] = count($directories) - 1;
return $count;
}
protected function getResultAsXml($result) {
$newline = "\n";
$newlineWithSpaces = sprintf("\n%4s",'');
$xml = '<?xml version="1.0" encoding="UTF-8"?>';
$xml.= $newline . '<phploc>';
if ($result['directories'] > 0) {
$xml.= $newlineWithSpaces . '<directories>' . $result['directories'] . '</directories>';
$xml.= $newlineWithSpaces . '<files>' . $result['files'] . '</files>';
}
$xml.= $newlineWithSpaces . '<loc>' . $result['loc'] . '</loc>';
if (isset($result['eloc'])) {
$xml.= $newlineWithSpaces . '<eloc>' . $result['eloc'] . '</eloc>';
}
$xml.= $newlineWithSpaces . '<cloc>' . $result['cloc'] . '</cloc>';
$xml.= $newlineWithSpaces . '<ncloc>' . $result['ncloc'] . '</ncloc>';
$xml.= $newlineWithSpaces . '<interfaces>' . $result['interfaces'] . '</interfaces>';
$xml.= $newlineWithSpaces . '<classes>' . $result['classes'] . '</classes>';
$xml.= $newlineWithSpaces . '<methods>' . $result['functions'] . '</methods>' . $newline;
$xml.= '</phploc>';
return $xml;
}
}
Hooking the phploc task into Phing
To use the task in your Phing builds simply copy it into the phing/tasks/my directory and make it available via the taskdef task. The next table shows the available task attributes and the values they can take to configure it's behaviour and output. As you will see it also provides the ability to generate reports in a XML format; I chose to implement this feature to have the possibilty to transform the report results into HTML documents by applying for example a XSLT stylesheet. This way they can provide more value to non-technical project members or can be made accessible in a CI system dashboard if desired.| Name | Type | Description | Default | Required |
|---|---|---|---|---|
| reportType | string | The type of the report. Available types are cli|txt|xml. | cli | No |
| reportName | string | The name of the report type without a file extension. | phploc-report | No |
| reportDirectory | string | The directory to write the report file to. | false | Yes, when report type txt or xml is defined. |
| file | string | The name of the file to check. | n/a | Yes, when no nested fileset is defined. |
| suffixes | string | A comma-separated list of file suffixes to check. | php | No |
Supported Nested Tags:
- fileset
<?xml version="1.0"?>
<project name="example" default="phploc" basedir=".">
<taskdef name="phploc" classname="phing.tasks.my.PHPLocTask" />
<target name="phploc">
<tstamp>
<format property="check.date.time" pattern="%Y%m%d-%H%M%S" locale="en_US"/>
</tstamp>
<phploc reportType="txt" reportName="${check.date.time}-report"
reportDirectory="phploc-reports">
<fileset dir=".">
<include name="**/*.php" />
<include name="*.php" />
</fileset>
</phploc>
</target>
</project>
Broadcasting blog post notifications to Twitter with Ruby and Rake 23 Jan 2009 8:08 PM (16 years ago)
During my latest blogging absence I had some time to tinker around with Ruby. For an introductory challenge I chose to implement a real life feature which currently isn't supported by Blogger.com and screams siren-like for an one-button automation: Broadcasting the latest blog entry to my Twitter account. As I didn't want to sign up for a Twitterfeed account and couldn't resort to the Twitter Tools plugin like WordPress users, I had to perform these broadcasting steps manually, until now. To see how this repetitive and time-stealing process was transformed into a semi-automated one by utilizing Ruby, a splash of Hpricot, Ruby's excellent Twitter Api wrapper and Rake, read on my dear.
Installing the required RubyGems
Prior to diving into the implementation details of the given scenario I had to install the required RubyGems like shown in the next console snippet. The installation of the twitter gem might take a while due to it's dependency on several other gems.sudo gem install hpricot rake twitter
Scraping the latest blog post details with Hpricot
The initial implementation step was to gather relevant metadata (Url, title and used tags) of the latest blog post. I first took the route to get it by grabbing the blog's RSS feed and extracting the metadata from there, but soon stumbled into problems getting an outdated feed from Feedburner. The next alternative was to scrape the needed metadata directly from the blog landing page. As I went this route before with the Zend_Dom_Query component of the Zend Framework I decided to use something similar from the Ruby toolbox. Some Google hops later I was sold to Hpricot, a HTML Parser for Ruby and as you can see in the first code snippet, showing an extract of the Rake file to come, this is done in just 13 lines of code.doc = Hpricot(open(blog_landing_page, scrape_options))
latest_post_url = doc.at('h3.post-title > a')['href']
latest_post_title = doc.at('h3.post-title > a').inner_html
label_doc = Hpricot(doc.search('span.post-labels').first.to_s)
label_links = label_doc.search('span.post-labels > a').each do |label_link|
label = label_link.inner_html.gsub(' ', '').downcase
if label.include?('/')
labels = label.split('/')
labels.each { |label| last_post_labels.push(label) }
else
last_post_labels.push(label)
end
end
Outstanding tasks
With the metadata available the oustanding tasks to implement were:- to get a short Url for the actual blog post by utilzing a public API of an Url shortening service i.e. is.gd
- to build the tweet to broadcast by injecting the available metadata into a tweet template
- to broadcast the notification tweet to the given Twitter account
- to log the broadcasted blog title to prevent spamming or duplication scenarios
require 'rubygems'
require 'hpricot'
require 'open-uri'
require 'twitter'
task :default do
Rake::Task['blog_utils:broadcast_notification'].invoke
end
namespace :blog_utils do
scrape_options = { 'UserAgent' => "Ruby/#{RUBY_VERSION}" }
blog_landing_page = 'http://raphaelstolt.blogspot.com'
latest_post_short_url, latest_post_url, latest_post_title = nil
notification_tweet = nil
last_post_labels = []
broadcast_log_file = File.dirname(__FILE__) + '/broadcasted_posts.log'
twitter_credentials = { :user => 'raphaelstolt', :pwd => 'thatsasecret'}
desc 'Scrape metadata of latest blog post from landing page'
task :scrape_actual_post_metadata do
doc = Hpricot(open(blog_landing_page, scrape_options))
latest_post_url = doc.at('h3.post-title > a')['href']
latest_post_title = doc.at('h3.post-title > a').inner_html
label_doc = Hpricot(doc.search('span.post-labels').first.to_s)
label_links = label_doc.search('span.post-labels > a').each do |label_link|
label = label_link.inner_html.gsub(' ', '').downcase
if label.include?('/')
labels = label.split('/')
labels.each { |label| last_post_labels.push(label) }
else
last_post_labels.push(label)
end
end
end
desc 'Shorten the Url of the latest blog post'
task :shorten_post_url => [:scrape_actual_post_metadata] do
raise_message = 'No Url for latest blog post available'
raise raise_message if latest_post_url.nil?
url_shorten_service_call = "http://is.gd/api.php?longurl=#{latest_post_url}"
latest_post_short_url = open(url_shorten_service_call, scrape_options).read
end
desc 'Check if generate shorten Url references the latest blog post url'
task :check_shorten_url_references_latest do
url_referenced_by_short_url = nil
open(latest_post_short_url, scrape_options) do |f|
url_referenced_by_short_url = f.base_uri.to_s
end
raise_message = "Generated short Url '#{latest_post_short_url}' does not"
raise_message << " reference actual blog post url '#{latest_post_url}'"
raise raise_message unless url_referenced_by_short_url.eql?(latest_post_url)
end
desc 'Check if latest blog post has already been broadcasted'
task :check_logged_broadcasts do
logged_broadcasts = []
if File.exist?(broadcast_log_file)
File.open(broadcast_log_file, 'r') do |f|
logged_broadcasts = f.readlines.collect { |line| line.chomp }
end
end
raise_message = "Blog post '#{latest_post_title}' has already been "
raise_message << "broadcasted"
raise raise_message if logged_broadcasts.include?(latest_post_title)
end
desc 'Build notification tweet by injecting scraped metadata into template'
task :build_notification_tweet => [:shorten_post_url,
:check_shorten_url_references_latest] do
raise_message = 'Required metadata to build tweet is not available'
raise raise_message if latest_post_title.nil? || latest_post_short_url.nil?
raise raise_message if last_post_labels.nil?
notification_tweet = "Published a new blog post '#{latest_post_title}' "
notification_tweet << "available at #{latest_post_short_url}."
raise_message = 'Broadcast for latest blog post exceeds 140 characters'
raise raise_message if notification_tweet.length > 140
last_post_labels.each do |tag|
notification_tweet << " ##{tag}" unless notification_tweet.length +
" ##{tag}".length > 140
end
end
desc 'Broadcast latest blog post notification to twitter'
task :broadcast_notification_to_twitter => [:build_notification_tweet,
:check_logged_broadcasts] do
raise_message = "Notification tweet to broadcast is not available"
raise raise_message if notification_tweet.nil?
puts "Broadcasting '#{notification_tweet}'"
http_auth = Twitter::HTTPAuth.new(twitter_credentials[:user], twitter_credentials[:pwd])
Twitter::Base.new(http_auth).update(notification_tweet)
#Twitter::Base.new(twitter_credentials[:user], twitter_credentials[:pwd]).post(notification_tweet)
Rake::Task['blog_utils:log_broadcast_title'].invoke
end
desc 'Log broadcasted blog post title'
task :log_broadcast_title do
puts "Logging latest post title to #{broadcast_log_file}"
File.open(broadcast_log_file, 'a') do |f|
f.puts latest_post_title
end
end
end
Putting the Rake task(s) to work
The next step was to put the Rakefile into my $HOME directory; and after publishing a new blog post I'm now able to broadcast an automated notification by firing up the console and calling the Rake task like shown next.sudo rake -f $HOME/Automations/Rakefile.rb blog_utils:broadcast_notificationAnd as I'm too lazy to type this lengthy command everytime I further added an alias to the $HOME/.profile file which allows me to call the task via the associated alias i.e. blogger2twitter shown in the .profile excerpt.
alias blogger2twitter='sudo rake -f $HOME/Automations/Rakefile.rb blog_utils:broadcast_notification'After running the Rake task against this blog post the notification gets added to the given Twitter timeline like shown in the outro image.
Installing Zend_Tool on Mac OS X 23 Jan 2009 4:19 AM (16 years ago)
Yesterday I decided to tiptoe into the development of custom Zend_Tool Providers as the introductional article series by Ralph Schindler motivated me to learn more about it and I already have some useful use cases on my mind. Therefor I prior had to install the Zend_Tool component and it's driving CLI scripts on my MacBook. The following brief instruction describes a possible approach that got me running in no time on a Mac OS X system. Once the Zend Framework has an official PEAR channel most of the forthcoming steps should be obsolete and entirely performed by the PEAR package installer command.
Fetching and installing the Zend_Tool component
First I tried to install the 1.8.0(devel) version of the Zend Framework via the pear.zfcampus.org PEAR channel but it currently only delivers the 1.7.3PL1(stable) package; even after switching the stability state of the PEAR config. To dodge the include_path setting hassle and for a further use when customizing other tools like Phing tasks I decided to keep the installed package.sudo pear channel-discover pear.zfcampus.orgThe next commands are showing the footwork I had to do to get the Zend_Tool component into the PEAR Zend Framework package installed in /opt/local/lib/php/Zend.
sudo pear install zfcampus/zf-devel
sudo svn co http://framework.zend.com/svn/framework/standard/incubator/library/Zend/Tool/ $HOME/Cos/Zend/Tool
sudo rsync -r --exclude=.svn $HOME/Cos/Zend/Tool /opt/local/lib/php/Zend
Putting the Zend_Tool CLI scripts to work
The next steps were to fetch the CLI scripts from the public Subversion repository and to link them into the system path /opt/local/bin as shown in the next commands.sudo svn co http://framework.zend.com/svn/framework/standard/incubator/bin $HOME/Cos/Zend/bin
sudo ln $HOME/Cos/Zend/bin/zf.sh /opt/local/bin/zf
sudo ln $HOME/Cos/Zend/bin/zf.php /opt/local/bin/zf.php
Checking the installation
With everything hopefully in place it was time to verify the success of the installation via the below stated provider action call; and as I got the version of the installed Zend Framework as a response of the executed action/command I'm good to go.zf show version
Rails for PHP Developers book review 13 Nov 2008 6:21 AM (16 years ago)
The e-book version of the Pragmatic Programmers release Rails for PHP Developers written by Derek DeVries and Mike Naberezny occupies now some of my scarce hard drive space for several months, and today I managed to hit the last page of it. In case you're interested in knowing if it's worthy to sacrifice some rare hard drive or bookshelf space for this book read on.
What's in it?
The book consists of three main parts which are addressing open-minded developers with a PHP background tempted to add the Ruby language and the thereupon built Rails 2.0 framework to their toolset.The first part introduces the classic and nowadays omnipresent MVC pattern, the concepts and conventions of Rails by converting a simple PHP newsletter application into a Rails based one. The follow-up chapters of the first part are covering the basics of the Ruby language by looking at known PHP language features and constructs, and how they translate to their Ruby counterparts. Reading these chapters you will get a thorough understanding of the Ruby language and be able to apply unique features like blocks or the reopening of existing classes. The communicated knowledge builds the foundation to accelerate the use and understanding of the Rails framework which is covered in-depth through-out the book's second part.
While teaming up with their imaginary buddy Joe the authors walk you through building a Rails user group application. The chapters of the second part are covering a lot of ground reaching from domain modeling, putting the particular MVC parts to work, ensuring quality by utilizing the Test::Unit library to finally deploying the application into a
The first two chapters of the final and reference part cover the differences and similarities between PHP and Ruby data structures, operations and language constructs. The final chapter of the book closes with a web development specific comparision of PHP constructs and approaches to the ones used by the Rails framework. The book is accompanied by a dedicated blog and a PHP to Rails online reference to satisfy severe thirst for more knowledge.
Conclusion
The book provides interested PHP developers a thorough introduction to the Ruby language and the Rails framework in a fluent and enjoyable writing tone. By implementing the example application of the second book part any decent PHP developer will derive a solid understanding of the Rails framework, he can build upon and that puts him in the position to make reasonable judgments for using/flaming it or not. IMHO this book is so far one of the best PHP related book releases of the out fading year 2008, and can be a real motivator to extend the just gained knowledge by diving deeper into the Ruby/Rails ocean.So be prepared to see one or another Ruby related post popping up in the future timeline of this blog; I just added another costly addiction to the medicine cupboard. Word!
Tinyizing URLs with Zend_Http_Client 31 Oct 2008 1:44 PM (16 years ago)
While doing some initial research for a blog related automation task to implement I learned some more about services which transform long URLs into short ones. The well-knownst of these services, due to the Twitter hype, is probably TinyURL which can be accessed via a classic webinterface or by calling a public API. In a recent blog post Dave Marshall outlined a quick workaround for tweeting via the Zend_Http_Client component which is a reasonable approach for calling services that aren't in the Zend Framework core yet like Zend_Service_Twitter or are not supported out of the box. Therefore this post will try to describe a Zend Framework way of creating tinyized URLs.
Getting tiny tiny y'all
According to Wikipedia there are numerous services available e.g. RubyUrl providing the same feature as TinyURL, so to be prepared for the future and thereby maybe violating the YAGNI principle I decided to declare a very basic interface first in case of switching the service provider someday.<?phpThe next code snippet shows the implementation for the TinyURL service programmed against the interface and hosting an additional alias method called tinyize which is simply wrapping the actual worker method. The service utilizes Zend_Http_Client by setting the endpoint of the service, transmitting a GET request parameterized with the URL to shorten against it and returning the response containing the tinyized URL.
/**
* 'Interface-level' PHPDoc Block
*/
interface Recordshelf_Service_UrlShortener_Interface
{
public function __construct($serviceEndpoint = '');
public function shortenize($url);
}
<?phpNow with everything hopefully operating smoothly it's time for a test-drive, yeah I'm lazy and cut that development approach called TDD, by creating a service instance and requesting a TinyURL for the Zend Framework website as shown in the outro listing.
require_once('Zend/Http/Client.php');
require_once('Recordshelf/Service/UrlShortener/Interface.php');
/**
* 'Class-level' PHPDoc Block
*/
class Recordshelf_Service_TinyUrl implements
Recordshelf_Service_UrlShortener_Interface
{
/**
* The service endpoint
*
* @var string
*/
private $_serviceEndpoint = null;
/**
* Recordshelf service tinyURL constructor
*
* @param string $serviceEndpoint
*/
public function __construct(
$serviceEndpoint = 'http://tinyurl.com/api-create.php')
{
$this->_serviceEndpoint = $serviceEndpoint;
}
/**
* Shortenizes a given Url
*
* @param string $url
* @return string
* @throws Exception
*/
public function shortenize($url) {
if (is_null($this->_serviceEndpoint)) {
throw new Exception('No service endpoint set');
}
$client = new Zend_Http_Client($this->_serviceEndpoint);
$client->setParameterGet('url', $url)
->setMethod(Zend_Http_Client::GET);
try {
$response = $client->request();
} catch (Exception $e) {
throw $e;
}
if (200 === $response->getStatus()) {
return $response->getBody();
} else {
throw new Exception($response->getStatus() . ": " .
$response->getMessage());
}
}
/**
* Alias method for the shortenize method
*
* @param string $url
* @throws Exception
* @see shortenize
*/
public function tinyize($url)
{
return $this->shortenize($url);
}
}
<?phpIn case off considering or favouring a more framework independent approach there are also other blends available like one via file_get_contents or via curl. Happy tinyizing!
$service = new Recordshelf_Service_TinyUrl();
$service->tinyize('http://framework.zend.com');
// => http://tinyurl.com/nf8kf