the whole shebang

This commit is contained in:
2014-11-25 16:42:40 +01:00
parent 7f74c0613e
commit ab1334c0cf
3686 changed files with 496409 additions and 1 deletions

View File

@@ -0,0 +1,2 @@
composer.lock
vendor

View File

@@ -0,0 +1,19 @@
Copyright (c) 2013 Michael Dowling <mtdowling@gmail.com> and contributors
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

View File

@@ -0,0 +1,108 @@
Class Preloader for PHP
=======================
This tool is used to generate a single PHP script containing all of the classes
required for a specific use case. Using a single compiled PHP script instead of relying on autoloading can help to improve the performance of specific use cases. For example, if your application executes the same bootstrap code on every request, then you could generate a preloader (the compiled output of this tool) to reduce the cost of autoloading the required classes over and over.
What it actually does
---------------------
This tool listens for each file that is autoloaded, creates a list of files, traverses the parsed PHP file using [PHPParser](https://github.com/nikic/PHP-Parser) and any visitors of a Config object, wraps the code of each file in a namespace block if necessary, and writes the contents of every autoloaded file (in order) to a single PHP file.
Notice
------
This tool should only be used for specific use cases. There is a tradeoff between preloading classes and autoloading classes. The point at which it is no longer beneficial to generate a preloader is application specific. You'll need to perform your own benchmarks to determine if this tool will speed up your application.
Installation
------------
Add the ClassPreloader as a dependency to your composer.json file:
```javascript
{
"require": {
"classpreloader/classpreloader": "1.0.0"
},
"config": {
"bin-dir": "bin"
}
}
```
Using the tool
--------------
You use the bin/classpreloader.php compile command with a few command line flags to generate a preloader.
`--config`: A CSV containing a list of files to combine into a classmap, or the full path to a PHP script that returns an array of classes or a `\ClassPreloader\Config` object.
`--output`: The path to the file to store the compiled PHP code. If the directory does not exist, the tool will attempt to create it.
`--fix_dir`: (defaults to 1) Set to 0 to not replace "__DIR__" constants with the actual directory of the original file.
`--fix_file`: (defaults to 1) Set to 0 to not replace "__FILE__" constants with the actual location of the original file.
Writing a config file
---------------------
Creating a PHP based configuration file is fairly simple. Just include the vendor/classpreloader/classpreloader/src/ClassPreloader/ClassLoader.php file and call the `ClassLoader::getIncludes()` method, passing a function as the only argument. This function should accept a `ClassLoader` object and register the passed in object's autoloader using `$loader->register()`. It is important to register the `ClassLoader` autoloader after all other autoloaders are registered.
An array or `\ClassPreloader\Config` must be returned from the config file. You can attach custom node visitors if you need to perform any sort of translation on each matching file before writing it to the output.
```php
<?php
// Here's an example of creating a preloader for using Amazon DynamoDB and the
// AWS SDK for PHP 2.
require __DIR__ . '/src/ClassPreloader/ClassLoader.php';
use ClassPreloader\ClassLoader;
$config = ClassLoader::getIncludes(function(ClassLoader $loader) {
require __DIR__ . '/vendor/autoload.php';
$loader->register();
$aws = Aws\Common\Aws::factory(array(
'key' => '***',
'secret' => '***',
'region' => 'us-east-1'
));
$client = $aws->get('dynamodb');
$client->listTables()->getAll();
});
// Add a regex filter that requires all classes to match the regex
// $config->addInclusiveFilter('/Foo/');
// Add a regex filter that requires that a class does not match the filter
// $config->addExclusiveFilter('/Foo/');
return $config;
```
You would then run the classpreloader.php script and pass in the full path to the above PHP script.
`php bin/classpreloader.php compile --config="/path/to/the_example.php" --output="/tmp/preloader.php"`
The above command will create a file in /tmp/preloader.php that contains every file that was autoloaded while running the snippet of code in the anonymous function. You would generate this file and include it in your production script.
Automating the process with Composer
------------------------------------
You can automate the process of creating preloaders using Composer's script functionality. For example, if you wanted to automatically create a preloader each time the AWS SDK for PHP is installed, you could define a script like the following in your composer.json file:
```javascript
{
"require": {
"classpreloader/classpreloader": "1.0.0"
},
"scripts": {
"post-autoload-dump": "php bin/classpreloader.php compile --config=/path/to/the_example.php --output=/path/to/preload.php"
},
"config": {
"bin-dir": "bin"
}
}
```
Using the above composer.json file, each time the project's autoloader is recreated using the install or update command, the classpreloader.php file will be executed. This script would generate a preload.php containing the classes required to run the previously demonstrated "the_example.php" configuration file.

View File

@@ -0,0 +1,10 @@
#! /usr/bin/env php
<?php
if (file_exists($autoloadPath = __DIR__ . '/../../autoload.php')) {
require_once $autoloadPath;
} else {
require_once __DIR__ . '/vendor/autoload.php';
}
$application = new ClassPreloader\Application();
$application->run();

View File

@@ -0,0 +1,30 @@
{
"name": "classpreloader/classpreloader",
"description":"Helps class loading performance by generating a single PHP file containing all of the autoloaded files for a specific use case",
"keywords":["autoload","class","preload"],
"license":"MIT",
"require":{
"php": ">=5.3.3",
"symfony/console": ">2.0",
"symfony/filesystem": ">2.0",
"symfony/finder": ">2.0",
"nikic/php-parser": "*"
},
"minimum-stability": "beta",
"autoload": {
"psr-0": {
"ClassPreloader": "src/"
}
},
"bin": ["classpreloader.php"],
"extra": {
"branch-alias": {
"dev-master": "1.0.0-dev"
}
}
}

View File

@@ -0,0 +1,33 @@
<?php
namespace ClassPreloader;
use Symfony\Component\Finder\Finder;
use Symfony\Component\Console\Application as BaseApplication;
/**
* ClassPreloader application CLI
*/
class Application extends BaseApplication
{
public function __construct()
{
parent::__construct('ClassPreloader');
// Create a finder to find each non-abstract command in the filesystem
$finder = new Finder();
$finder->files()
->in(__DIR__ . '/Command')
->notName('Abstract*')
->name('*.php');
// Add each command to the CLI
foreach ($finder as $file) {
$filename = str_replace('\\', '/', $file->getRealpath());
$pos = strrpos($filename, '/ClassPreloader/') + strlen('/ClassPreloader/');
$class = __NAMESPACE__ . '\\'
. substr(str_replace('/', '\\', substr($filename, $pos)), 0, -4);
$this->add(new $class());
}
}
}

View File

@@ -0,0 +1,87 @@
<?php
namespace ClassPreloader;
/**
* Maintains a list of classes using a sort of doubly-linked list
*/
class ClassList
{
/**
* @var ClassNode The head node of the list
*/
protected $head;
/**
* @var ClassNode The current node of the list
*/
protected $current;
public function __construct()
{
$this->clear();
}
/**
* Clear the contents of the list and reset the head node and current node
*/
public function clear()
{
$this->head = new ClassNode(null);
$this->current = $this->head;
}
/**
* Traverse to the next node in the list
*/
public function next()
{
if (isset($this->current->next)) {
$this->current = $this->current->next;
} else {
$this->current->next = new ClassNode(null, $this->current);
$this->current = $this->current->next;
}
}
/**
* Insert a value at the current position in the list. Any currently set
* value at this position will be pushed back in the list after the new
* value
*
* @param mixed $value Value to insert
*/
public function push($value)
{
if (!$this->current->value) {
$this->current->value = $value;
} else {
$temp = $this->current;
$this->current = new ClassNode($value, $temp->prev);
$this->current->next = $temp;
$temp->prev = $this->current;
if ($temp === $this->head) {
$this->head = $this->current;
} else {
$this->current->prev->next = $this->current;
}
}
}
/**
* Traverse the ClassList and return a list of classes
*
* @return array
*/
public function getClasses()
{
$classes = array();
$current = $this->head;
while ($current && $current->value) {
$classes[] = $current->value;
$current = $current->next;
}
return array_filter($classes);
}
}

View File

@@ -0,0 +1,112 @@
<?php
namespace ClassPreloader;
require_once __DIR__ . '/ClassNode.php';
require_once __DIR__ . '/ClassList.php';
/**
* Creates an autoloader that intercepts and keeps track of each include in
* order that files must be included. This autoloader proxies to all other
* underlying autoloaders.
*/
class ClassLoader
{
/**
* @var ClassList List of loaded classes
*/
public $classList;
/**
* Create the dependency list
*/
public function __construct()
{
$this->classList = new ClassList();
}
/**
* Wrap a block of code in the autoloader and get a list of loaded classes
*
* @param \Callable $func Callable function
*
* @return Config
*/
public static function getIncludes($func)
{
$loader = new self();
call_user_func($func, $loader);
$loader->unregister();
$config = new Config();
foreach ($loader->getFilenames() as $file) {
$config->addFile($file);
}
return $config;
}
/**
* Registers this instance as an autoloader.
*
* @param bool $prepend Whether to prepend the autoloader or not
*/
public function register()
{
spl_autoload_register(array($this, 'loadClass'), true, true);
}
/**
* Unregisters this instance as an autoloader.
*/
public function unregister()
{
spl_autoload_unregister(array($this, 'loadClass'));
}
/**
* Loads the given class or interface.
*
* @param string $class The name of the class
* @return bool|null True, if loaded
*/
public function loadClass($class)
{
foreach (spl_autoload_functions() as $func) {
if (is_array($func) && $func[0] === $this) {
continue;
}
$this->classList->push($class);
if (call_user_func($func, $class)) {
break;
}
}
$this->classList->next();
return true;
}
/**
* Get an array of loaded file names in order of loading
*
* @return array
*/
public function getFilenames()
{
$files = array();
foreach ($this->classList->getClasses() as $class) {
// Push interfaces before classes if not already loaded
$r = new \ReflectionClass($class);
foreach ($r->getInterfaces() as $inf) {
$name = $inf->getFileName();
if ($name && !in_array($name, $files)) {
$files[] = $name;
}
}
$files[] = $r->getFileName();
}
return $files;
}
}

View File

@@ -0,0 +1,36 @@
<?php
namespace ClassPreloader;
/**
* A simple ClassNode that contains a value, previous, and next pointers
*/
class ClassNode
{
/**
* @var ClassNode|null Next node pointer
*/
public $next;
/**
* @var ClassNode|null Previous node pointer
*/
public $prev;
/**
* @var mixed Value of the ClassNode
*/
public $value;
/**
* Create a new ClassNode
*
* @param mixed $value Value of the class node
* @param ClassNode $prev Previous node pointer
*/
public function __construct($value = null, $prev = null)
{
$this->value = $value;
$this->prev = $prev;
}
}

View File

@@ -0,0 +1,213 @@
<?php
namespace ClassPreloader\Command;
use ClassPreloader\Config;
use ClassPreloader\Parser\DirVisitor;
use ClassPreloader\Parser\NodeTraverser;
use ClassPreloader\Parser\FileVisitor;
use Symfony\Component\Filesystem\Filesystem;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Input\InputOption;
use Symfony\Component\Console\Output\OutputInterface;
use Symfony\Component\Console\Command\Command;
class PreCompileCommand extends Command
{
protected $input;
protected $output;
protected $printer;
protected $traverser;
protected $parser;
public function __construct()
{
parent::__construct();
$this->printer = new \PHPParser_PrettyPrinter_Zend();
$this->parser = new \PHPParser_Parser(new \PHPParser_Lexer());
}
/**
* {@inheritdoc}
*/
protected function configure()
{
parent::configure();
$this->setName('compile')
->setDescription('Compiles classes into a single file')
->addOption('config', null, InputOption::VALUE_REQUIRED, 'CSV of filenames to load, or the path to a PHP script that returns an array of file names')
->addOption('output', null, InputOption::VALUE_REQUIRED)
->addOption('fix_dir', null, InputOption::VALUE_REQUIRED, 'Convert __DIR__ constants to the original directory of a file', 1)
->addOption('fix_file', null, InputOption::VALUE_REQUIRED, 'Convert __FILE__ constants to the original path of a file', 1)
->addOption('strip_comments', null, InputOption::VALUE_REQUIRED, 'Set to 1 to strip comments from each source file', 0)
->setHelp(<<<EOF
The <info>%command.name%</info> command iterates over each script, normalizes
the file to be wrapped in namespaces, and combines each file into a single PHP
file.
EOF
);
}
/**
* Get the node traverser used by the command
*
* @return NodeTraverser
*/
protected function getTraverser()
{
if (!$this->traverser) {
$this->traverser = new NodeTraverser();
if ($this->input->getOption('fix_dir')) {
$this->traverser->addVisitor(new DirVisitor($file));
}
if ($this->input->getOption('fix_file')) {
$this->traverser->addVisitor(new FileVisitor($file));
}
}
return $this->traverser;
}
/**
* Get a pretty printed string of code from a file while applying visitors
*
* @param string $file Name of the file to get code from
* @param NodeTraverser $traverser Node traverser
*
* @return string
*/
protected function getCode($file)
{
if (!is_readable($file)) {
throw new \RuntimeException("Cannot open {$file} for reading");
}
if ($this->input->getOption('strip_comments')) {
$content = php_strip_whitespace($file);
} else {
$content = file_get_contents($file);
}
$stmts = $this->getTraverser()
->traverseFile($this->parser->parse($content), $file);
$pretty = $this->printer->prettyPrint($stmts);
// Remove the open PHP tag
if (substr($pretty, 6) == "<?php\n") {
$pretty = substr($pretty, 7);
}
// Add a wrapping namespace if needed
if (false === strpos($pretty, 'namespace ')) {
$pretty = "namespace {\n" . $pretty . "\n}\n";
}
return $pretty;
}
/**
* Validate the command options
*/
protected function validateCommand()
{
if (!$this->input->getOption('output')) {
throw new \InvalidArgumentException('An output option is required');
}
if (!$this->input->getOption('config')) {
throw new \InvalidArgumentException('A config option is required');
}
}
/**
* Get a list of files in order
*
* @param mixed $config Configuration option
*
* @return array
*/
protected function getFileList($config)
{
$this->output->writeln('> Loading configuration file');
$filesystem = new Filesystem();
if (strpos($config, ',')) {
return array_filter(explode(',', $config));
}
// Ensure absolute paths are resolved
if (!$filesystem->isAbsolutePath($config)) {
$config = getcwd() . '/' . $config;
}
// Ensure that the config file exists
if (!file_exists($config)) {
throw new \InvalidArgumentException(sprintf('Configuration file "%s" does not exist.', $config));
}
$result = require $config;
if ($result instanceof Config) {
foreach ($result->getVisitors() as $visitor) {
$this->getTraverser()->addVisitor($visitor);
}
return $result;
} elseif (is_array($result)) {
return $result;
}
throw new \InvalidArgumentException(
'Config must return an array of filenames or a Config object'
);
}
/**
* Prepare the output file and directory
*
* @param string $outputFile The full path to the output file
*/
protected function prepareOutput($outputFile)
{
$dir = dirname($outputFile);
if (!is_dir($dir) && !mkdir($dir, 0777, true)) {
throw new \RuntimeException('Unable to create directory ' . $dir);
}
}
/**
* {@inheritdoc}
*/
protected function execute(InputInterface $input, OutputInterface $output)
{
$this->input = $input;
$this->output = $output;
$this->validateCommand();
$outputFile = $this->input->getOption('output');
$config = $this->input->getOption('config');
$files = $this->getFileList($config);
$output->writeLn('- Found ' . count($files) . ' files');
// Make sure that the output dir can be used or create it
$this->prepareOutput($outputFile);
if (!$handle = fopen($input->getOption('output'), 'w')) {
throw new \RuntimeException(
"Unable to open {$outputFile} for writing"
);
}
// Write the first line of the output
fwrite($handle, "<?php\n");
$output->writeln('> Compiling classes');
foreach ($files as $file) {
$this->output->writeln('- Writing ' . $file);
fwrite($handle, $this->getCode($file) . "\n");
}
fclose($handle);
$output->writeln("> Compiled loader written to {$outputFile}");
$output->writeln('- ' . (round(filesize($outputFile) / 1024)) . ' kb');
}
}

View File

@@ -0,0 +1,133 @@
<?php
namespace ClassPreloader;
use Parser\AbstractNodeVisitor;
/**
* Class loader configuration object
*/
class Config implements \IteratorAggregate
{
/**
* @var array Array of AbstractNodeVisitor objects that visit nodes
*/
protected $visitors = array();
/**
* @var array Array of file names
*/
protected $filenames = array();
/**
* @var array Array of exclusive filters
*/
protected $exclusiveFilters = array();
/**
* @var array Array of inclusive filters
*/
protected $inclusiveFilters = array();
/**
* Set the filenames owned by the config
*
* @param array $filenames File name
*
* @return self
*/
public function addFile($filename)
{
$this->filenames[] = $filename;
return $this;
}
/**
* Get an array of file names that satisfy any added filters
*
* @return array
*/
public function getFilenames()
{
$filenames = array();
foreach ($this->filenames as $f) {
foreach ($this->inclusiveFilters as $filter) {
if (!preg_match($filter, $f)) {
continue 2;
}
}
foreach ($this->exclusiveFilters as $filter) {
if (preg_match($filter, $f)) {
continue 2;
}
}
$filenames[] = $f;
}
return $filenames;
}
/**
* Get an iterator for the filenames
*
* @return \ArrayIterator
*/
public function getIterator()
{
return new \ArrayIterator($this->getFilenames());
}
/**
* Add a filter used to filter out classes matching a specific pattern
*
* @param string $pattern Regular expression pattern
*
* @return self
*/
public function addExclusiveFilter($pattern)
{
$this->exclusiveFilters[] = $pattern;
return $this;
}
/**
* Add a filter used to grab only file names matching the pattern
*
* @param string $pattern Regular expression pattern
*
* @return self
*/
public function addInclusiveFilter($pattern)
{
$this->inclusiveFilters[] = $pattern;
return $this;
}
/**
* Add a visitor that will visit each node when traversing the node list
* of each file.
*
* @param AbstractNodeVisitor $visitor Node visitor
*
* @return self
*/
public function addVisitor(AbstractNodeVisitor $visitor)
{
$this->visitors[] = $visitor;
return $this;
}
/**
* Get an array of node visitors
*
* @return array
*/
public function getVisitors()
{
return $this->visitors;
}
}

View File

@@ -0,0 +1,48 @@
<?php
namespace ClassPreloader\Parser;
/**
* Abstract node visitor used to track the filename
*/
abstract class AbstractNodeVisitor extends \PHPParser_NodeVisitorAbstract
{
/**
* @var string Current file being parsed
*/
protected $filename = '';
/**
* Set the full path to the current file being parsed
*
* @param string $filename Filename being parser
*
* @return self
*/
public function setFilename($filename)
{
$this->filename = $filename;
return $this;
}
/**
* Get the full path to the current file being parsed
*
* @return string
*/
public function getFilename()
{
return $this->filename;
}
/**
* Get the directory of the current file being parsed
*
* @return string
*/
public function getDir()
{
return dirname($this->getFilename());
}
}

View File

@@ -0,0 +1,16 @@
<?php
namespace ClassPreloader\Parser;
/**
* Finds all references to __DIR__ and replaces them with the actual directory
*/
class DirVisitor extends AbstractNodeVisitor
{
public function enterNode(\PHPParser_Node $node)
{
if ($node instanceof \PHPParser_Node_Scalar_DirConst) {
return new \PHPParser_Node_Scalar_String($this->getDir());
}
}
}

View File

@@ -0,0 +1,16 @@
<?php
namespace ClassPreloader\Parser;
/**
* Finds all references to __FILE__ and replaces them with the actual file path
*/
class FileVisitor extends AbstractNodeVisitor
{
public function enterNode(\PHPParser_Node $node)
{
if ($node instanceof \PHPParser_Node_Scalar_FileConst) {
return new \PHPParser_Node_Scalar_String($this->getFilename());
}
}
}

View File

@@ -0,0 +1,21 @@
<?php
namespace ClassPreloader\Parser;
/**
* Allows a filename to be set when visiting
*/
class NodeTraverser extends \PHPParser_NodeTraverser
{
public function traverseFile(array $nodes, $filename)
{
// Set the correct state on each visitor
foreach ($this->visitors as $visitor) {
if ($visitor instanceof AbstractNodeVisitor) {
$visitor->setFilename($filename);
}
}
return $this->traverse($nodes);
}
}