the whole shebang

This commit is contained in:
2014-11-25 16:42:40 +01:00
parent 7f74c0613e
commit ab1334c0cf
3686 changed files with 496409 additions and 1 deletions

View File

@@ -0,0 +1,81 @@
Introduction
============
This project is a PHP 5.5 (and older) parser **written in PHP itself**.
What is this for?
-----------------
A parser is useful for [static analysis][0] and manipulation of code and basically any other
application dealing with code programmatically. A parser constructs an [Abstract Syntax Tree][1]
(AST) of the code and thus allows dealing with it in an abstract and robust way.
There are other ways of dealing with source code. One that PHP supports natively is using the
token stream generated by [`token_get_all`][2]. The token stream is much more low level than
the AST and thus has different applications: It allows to also analyze the exact formatting of
a file. On the other hand the token stream is much harder to deal with for more complex analysis.
For example an AST abstracts away the fact that in PHP variables can be written as `$foo`, but also
as `$$bar`, `${'foobar'}` or even `${!${''}=barfoo()}`. You don't have to worry about recognizing
all the different syntaxes from a stream of tokens.
Another questions is: Why would I want to have a PHP parser *written in PHP*? Well, PHP might not be
a language especially suited for fast parsing, but processing the AST is much easier in PHP than it
would be in other, faster languages like C. Furthermore the people most probably wanting to do
programmatic PHP code analysis are incidentally PHP developers, not C developers.
What can it parse?
------------------
The parser uses a PHP 5.5 compliant grammar, which is backwards compatible with at least PHP 5.4, PHP 5.3
and PHP 5.2 (and maybe older).
As the parser is based on the tokens returned by `token_get_all` (which is only able to lex the PHP
version it runs on), additionally a wrapper for emulating new tokens from 5.3, 5.4 and 5.5 is provided. This
allows to parse PHP 5.5 source code running on PHP 5.2, for example. This emulation is very hacky and not
yet perfect, but it should work well on any sane code.
What output does it produce?
----------------------------
The parser produces an [Abstract Syntax Tree][1] (AST) also known as a node tree. How this looks like
can best be seen in an example. The program `<?php echo 'Hi', 'World';` will give you a node tree
roughly looking like this:
```
array(
0: Stmt_Echo(
exprs: array(
0: Scalar_String(
value: Hi
)
1: Scalar_String(
value: World
)
)
)
)
```
This matches the semantics the program had: An echo statement, which takes two strings as expressions,
with the values `Hi` and `World!`.
You can also see that the AST does not contain any whitespace information (but most comments are saved).
So using it for formatting analysis is not possible.
What else can it do?
--------------------
Apart from the parser itself this package also bundles support for some other, related features:
* Support for pretty printing, which is the act of converting an AST into PHP code. Please note
that "pretty printing" does not imply that the output is especially pretty. It's just how it's
called ;)
* Support for serializing and unserializing the node tree to XML
* Support for dumping the node tree in a human readable form (see the section above for an
example of how the output looks like)
* Infrastructure for traversing and changing the AST (node traverser and node visitors)
* A node visitor for resolving namespaced names
[0]: http://en.wikipedia.org/wiki/Static_program_analysis
[1]: http://en.wikipedia.org/wiki/Abstract_syntax_tree
[2]: http://php.net/token_get_all

View File

@@ -0,0 +1,48 @@
Installation
============
There are multiple ways to include the PHP parser into your project:
Installing from the Zip- or Tarball
-----------------------------------
Download the latest version from [the download page][2], unpack it and move the files somewhere into your project.
Installing via Composer
-----------------------
Create a `composer.json` file in your project root and use it to define your dependencies:
{
"require": {
"nikic/php-parser": "0.9.4"
}
}
Then install Composer in your project (or [download the composer.phar][1] directly):
curl -s http://getcomposer.org/installer | php
And finally ask Composer to install the dependencies:
php composer.phar install
Installing as a PEAR package
----------------------------
Run the following two commands:
pear channel-discover nikic.github.com/pear
pear install nikic/PHPParser-0.9.4
Installing as a Git Submodule
-----------------------------
Run the following command to install the parser into the `vendor/PHP-Parser` folder:
git submodule add git://github.com/nikic/PHP-Parser.git vendor/PHP-Parser
[1]: http://getcomposer.org/composer.phar
[2]: https://github.com/nikic/PHP-Parser/tags

View File

@@ -0,0 +1,394 @@
Usage of basic components
=========================
This document explains how to use the parser, the pretty printer and the node traverser.
Bootstrapping
-------------
The library needs to register a class autoloader; this is done by including the
`bootstrap.php` file:
```php
<?php
require 'path/to/PHP-Parser/lib/bootstrap.php';
```
Additionally you may want to set the `xdebug.max_nesting_level` ini option to a higher value:
```php
<?php
ini_set('xdebug.max_nesting_level', 2000);
```
This ensures that there will be no errors when traversing highly nested node trees.
Parsing
-------
In order to parse some source code you first have to create a `PHPParser_Parser` object (which
needs to be passed a `PHPParser_Lexer` instance) and then pass the code (including `<?php` opening
tags) to the `parse` method. If a syntax error is encountered `PHPParser_Error` is thrown, so this
exception should be `catch`ed.
```php
<?php
$code = '<?php // some code';
$parser = new PHPParser_Parser(new PHPParser_Lexer);
try {
$stmts = $parser->parse($code);
} catch (PHPParser_Error $e) {
echo 'Parse Error: ', $e->getMessage();
}
```
The `parse` method will return an array of statement nodes (`$stmts`).
### Emulative lexer
Instead of `PHPParser_Lexer` one can also use `PHPParser_Lexer_Emulative`. This class will emulate tokens
of newer PHP versions and as such allow parsing PHP 5.5 on PHP 5.2, for example. So if you want to parse
PHP code of newer versions than the one you are running, you should use the emulative lexer.
Node tree
---------
If you use the above code with `$code = "<?php echo 'Hi ', hi\\getTarget();"` the parser will
generate a node tree looking like this:
```
array(
0: Stmt_Echo(
exprs: array(
0: Scalar_String(
value: Hi
)
1: Expr_FuncCall(
name: Name(
parts: array(
0: hi
1: getTarget
)
)
args: array(
)
)
)
)
)
```
Thus `$stmts` will contain an array with only one node, with this node being an instance of
`PHPParser_Node_Stmt_Echo`.
As PHP is a large language there are approximately 140 different nodes. In order to make work
with them easier they are grouped into three categories:
* `PHPParser_Node_Stmt`s are statement nodes, i.e. language constructs that do not return
a value and can not occur in an expression. For example a class definition is a statement.
It doesn't return a value and you can't write something like `func(class A {});`.
* `PHPParser_Node_Expr`s are expression nodes, i.e. language constructs that return a value
and thus can occur in other expressions. Examples of expressions are `$var`
(`PHPParser_Node_Expr_Variable`) and `func()` (`PHPParser_Node_Expr_FuncCall`).
* `PHPParser_Node_Scalar`s are nodes representing scalar values, like `'string'`
(`PHPParser_Node_Scalar_String`), `0` (`PHPParser_Node_Scalar_LNumber`) or magic constants
like `__FILE__` (`PHPParser_Node_Scalar_FileConst`). All `PHPParser_Node_Scalar`s extend
`PHPParser_Node_Expr`, as scalars are expressions, too.
* There are some nodes not in either of these groups, for example names (`PHPParser_Node_Name`)
and call arguments (`PHPParser_Node_Arg`).
Every node has a (possibly zero) number of subnodes. You can access subnodes by writing
`$node->subNodeName`. The `Stmt_Echo` node has only one subnode `exprs`. So in order to access it
in the above example you would write `$stmts[0]->exprs`. If you wanted to access name of the function
call, you would write `$stmts[0]->exprs[1]->name`.
All nodes also define a `getType()` method that returns the node type (the type is the class name
without the `PHPParser_Node_` prefix).
It is possible to associate custom metadata with a node using the `setAttribute()` method. This data
can then be retrieved using `hasAttribute()`, `getAttribute()` and `getAttributes()`.
By default the lexer adds the `startLine`, `endLine` and `comments` attributes. `comments` is an array
of `PHPParser_Comment[_Doc]` instances.
The start line can also be accessed using `getLine()`/`setLine()` (instead of `getAttribute('startLine')`).
The last doc comment from the `comments` attribute can be obtained using `getDocComment()`.
Pretty printer
--------------
The pretty printer component compiles the AST back to PHP code. As the parser does not retain formatting
information the formatting is done using a specified scheme. Currently there is only one scheme available,
namely `PHPParser_PrettyPrinter_Default`.
```php
<?php
$code = "<?php echo 'Hi ', hi\\getTarget();";
$parser = new PHPParser_Parser(new PHPParser_Lexer);
$prettyPrinter = new PHPParser_PrettyPrinter_Default;
try {
// parse
$stmts = $parser->parse($code);
// change
$stmts[0] // the echo statement
->exprs // sub expressions
[0] // the first of them (the string node)
->value // it's value, i.e. 'Hi '
= 'Hallo '; // change to 'Hallo '
// pretty print
$code = '<?php ' . $prettyPrinter->prettyPrint($stmts);
echo $code;
} catch (PHPParser_Error $e) {
echo 'Parse Error: ', $e->getMessage();
}
```
The above code will output:
<?php echo 'Hallo ', hi\getTarget();
As you can see the source code was first parsed using `PHPParser_Parser->parse`, then changed and then
again converted to code using `PHPParser_PrettyPrinter_Default->prettyPrint`.
The `prettyPrint` method pretty prints a statements array. It is also possible to pretty print only a
single expression using `prettyPrintExpr`.
Node traversation
-----------------
The above pretty printing example used the fact that the source code was known and thus it was easy to
write code that accesses a certain part of a node tree and changes it. Normally this is not the case.
Usually you want to change / analyze code in a generic way, where you don't know how the node tree is
going to look like.
For this purpose the parser provides a component for traversing and visiting the node tree. The basic
structure of a program using this `PHPParser_NodeTraverser` looks like this:
```php
<?php
$code = "<?php // some code";
$parser = new PHPParser_Parser(new PHPParser_Lexer);
$traverser = new PHPParser_NodeTraverser;
$prettyPrinter = new PHPParser_PrettyPrinter_Default;
// add your visitor
$traverser->addVisitor(new MyNodeVisitor);
try {
// parse
$stmts = $parser->parse($code);
// traverse
$stmts = $traverser->traverse($stmts);
// pretty print
$code = '<?php ' . $prettyPrinter->prettyPrint($stmts);
echo $code;
} catch (PHPParser_Error $e) {
echo 'Parse Error: ', $e->getMessage();
}
```
A same node visitor for this code might look like this:
```php
<?php
class MyNodeVisitor extends PHPParser_NodeVisitorAbstract
{
public function leaveNode(PHPParser_Node $node) {
if ($node instanceof PHPParser_Node_Scalar_String) {
$node->value = 'foo';
}
}
}
```
The above node visitor would change all string literals in the program to `'foo'`.
All visitors must implement the `PHPParser_NodeVisitor` interface, which defined the following four
methods:
public function beforeTraverse(array $nodes);
public function enterNode(PHPParser_Node $node);
public function leaveNode(PHPParser_Node $node);
public function afterTraverse(array $nodes);
The `beforeTraverse` method is called once before the traversal begins and is passed the nodes the
traverser was called with. This method can be used for resetting values before traversation or
preparing the tree for traversal.
The `afterTraverse` method is similar to the `beforeTraverse` method, with the only difference that
it is called once after the traversal.
The `enterNode` and `leaveNode` methods are called on every node, the former when it is entered,
i.e. before its subnodes are traversed, the latter when it is left.
All four methods can either return the changed node or not return at all (i.e. `null`) in which
case the current node is not changed. The `leaveNode` method can furthermore return two special
values: If `false` is returned the current node will be removed from the parent array. If an `array`
is returned the current node will be merged into the parent array at the offset of the current node.
I.e. if in `array(A, B, C)` the node `B` should be replaced with `array(X, Y, Z)` the result will be
`array(A, X, Y, Z, C)`.
Instead of manually implementing the `NodeVisitor` interface you can also extend the `NodeVisitorAbstract`
class, which will define empty default implementations for all the above methods.
The NameResolver node visitor
-----------------------------
One visitor is already bundled with the package: `PHPParser_NodeVisitor_NameResolver`. This visitor
helps you work with namespaced code by trying to resolve most names to fully qualified ones.
For example, consider the following code:
use A as B;
new B\C();
In order to know that `B\C` really is `A\C` you would need to track aliases and namespaces yourself.
The `NameResolver` takes care of that and resolves names as far as possible.
After running it most names will be fully qualified. The only names that will stay unqualified are
unqualified function and constant names. These are resolved at runtime and thus the visitor can't
know which function they are referring to. In most cases this is a non-issue as the global functions
are meant.
Also the `NameResolver` adds a `namespacedName` subnode to class, function and constant declarations
that contains the namespaced name instead of only the shortname that is available via `name`.
Example: Converting namespaced code to pseudo namespaces
--------------------------------------------------------
A small example to understand the concept: We want to convert namespaced code to pseudo namespaces
so it works on 5.2, i.e. names like `A\\B` should be converted to `A_B`. Note that such conversions
are fairly complicated if you take PHP's dynamic features into account, so our conversion will
assume that no dynamic features are used.
We start off with the following base code:
```php
<?php
const IN_DIR = '/some/path';
const OUT_DIR = '/some/other/path';
// use the emulative lexer here, as we are running PHP 5.2 but want to parse PHP 5.3
$parser = new PHPParser_Parser(new PHPParser_Lexer_Emulative);
$traverser = new PHPParser_NodeTraverser;
$prettyPrinter = new PHPParser_PrettyPrinter_Default;
$traverser->addVisitor(new PHPParser_NodeVisitor_NameResolver); // we will need resolved names
$traverser->addVisitor(new NodeVisitor_NamespaceConverter); // our own node visitor
// iterate over all .php files in the directory
$files = new RecursiveIteratorIterator(new RecursiveDirectoryIterator(IN_DIR));
$files = new RegexIterator($files, '/\.php$/');
foreach ($files as $file) {
try {
// read the file that should be converted
$code = file_get_contents($file);
// parse
$stmts = $parser->parse($code);
// traverse
$stmts = $traverser->traverse($stmts);
// pretty print
$code = '<?php ' . $prettyPrinter->prettyPrint($stmts);
// write the converted file to the target directory
file_put_contents(
substr_replace($file->getPathname(), OUT_DIR, 0, strlen(IN_DIR)),
$code
);
} catch (PHPParser_Error $e) {
echo 'Parse Error: ', $e->getMessage();
}
}
```
Now lets start with the main code, the `NodeVisitor_NamespaceConverter`. One thing it needs to do
is convert `A\\B` style names to `A_B` style ones.
```php
<?php
class NodeVisitor_NamespaceConverter extends PHPParser_NodeVisitorAbstract
{
public function leaveNode(PHPParser_Node $node) {
if ($node instanceof PHPParser_Node_Name) {
return new PHPParser_Node_Name($node->toString('_'));
}
}
}
```
The above code profits from the fact that the `NameResolver` already resolved all names as far as
possible, so we don't need to do that. All the need to create a string with the name parts separated
by underscores instead of backslashes. This is what `$node->toString('_')` does. (If you want to
create a name with backslashes either write `$node->toString()` or `(string) $node`.) Then we create
a new name from the string and return it. Returning a new node replaces the old node.
Another thing we need to do is change the class/function/const declarations. Currently they contain
only the shortname (i.e. the last part of the name), but they need to contain the complete class
name:
```php
<?php
class NodeVisitor_NamespaceConverter extends PHPParser_NodeVisitorAbstract
{
public function leaveNode(PHPParser_Node $node) {
if ($node instanceof PHPParser_Node_Name) {
return new PHPParser_Node_Name($node->toString('_'));
} elseif ($node instanceof PHPParser_Node_Stmt_Class
|| $node instanceof PHPParser_Node_Stmt_Interface
|| $node instanceof PHPParser_Node_Stmt_Function) {
$node->name = $node->namespacedName->toString('_');
} elseif ($node instanceof PHPParser_Node_Stmt_Const) {
foreach ($node->consts as $const) {
$const->name = $const->namespacedName->toString('_');
}
}
}
}
```
There is not much more to it than converting the namespaced name to string with `_` as separator.
The last thing we need to do is remove the `namespace` and `use` statements:
```php
<?php
class NodeVisitor_NamespaceConverter extends PHPParser_NodeVisitorAbstract
{
public function leaveNode(PHPParser_Node $node) {
if ($node instanceof PHPParser_Node_Name) {
return new PHPParser_Node_Name($node->toString('_'));
} elseif ($node instanceof PHPParser_Node_Stmt_Class
|| $node instanceof PHPParser_Node_Stmt_Interface
|| $node instanceof PHPParser_Node_Stmt_Function) {
$node->name = $node->namespacedName->toString('_');
} elseif ($node instanceof PHPParser_Node_Stmt_Const) {
foreach ($node->consts as $const) {
$const->name = $const->namespacedName->toString('_');
}
} elseif ($node instanceof PHPParser_Node_Stmt_Namespace) {
// returning an array merges is into the parent array
return $node->stmts;
} elseif ($node instanceof PHPParser_Node_Stmt_Use) {
// returning false removed the node altogether
return false;
}
}
}
```
That's all.

View File

@@ -0,0 +1,201 @@
Other node tree representations
===============================
It is possible to convert the AST in several textual representations, which serve different uses.
Simple serialization
--------------------
It is possible to serialize the node tree using `serialize()` and also unserialize it using
`unserialize()`. The output is not human readable and not easily processable from anything
but PHP, but it is compact and generates fast. The main application thus is in caching.
Human readable dumping
----------------------
Furthermore it is possible to dump nodes into a human readable form using the `dump` method of
`PHPParser_NodeDumper`. This can be used for debugging.
```php
<?php
$code = <<<'CODE'
<?php
function printLine($msg) {
echo $msg, "\n";
}
printLine('Hallo World!!!');
CODE;
$parser = new PHPParser_Parser(new PHPParser_Lexer);
$nodeDumper = new PHPParser_NodeDumper;
try {
$stmts = $parser->parse($code);
echo '<pre>' . htmlspecialchars($nodeDumper->dump($stmts)) . '</pre>';
} catch (PHPParser_Error $e) {
echo 'Parse Error: ', $e->getMessage();
}
```
The above output will have an output looking roughly like this:
```
array(
0: Stmt_Function(
byRef: false
params: array(
0: Param(
name: msg
default: null
type: null
byRef: false
)
)
stmts: array(
0: Stmt_Echo(
exprs: array(
0: Expr_Variable(
name: msg
)
1: Scalar_String(
value:
)
)
)
)
name: printLine
)
1: Expr_FuncCall(
name: Name(
parts: array(
0: printLine
)
)
args: array(
0: Arg(
value: Scalar_String(
value: Hallo World!!!
)
byRef: false
)
)
)
)
```
Serialization to XML
--------------------
It is also possible to serialize the node tree to XML using `PHPParser_Serializer_XML->serialize()`
and to unserialize it using `PHPParser_Unserializer_XML->unserialize()`. This is useful for
interfacing with other languages and applications or for doing transformation using XSLT.
```php
<?php
$code = <<<'CODE'
<?php
function printLine($msg) {
echo $msg, "\n";
}
printLine('Hallo World!!!');
CODE;
$parser = new PHPParser_Parser(new PHPParser_Lexer);
$serializer = new PHPParser_Serializer_XML;
try {
$stmts = $parser->parse($code);
echo '<pre>' . htmlspecialchars($serializer->serialize($stmts)) . '</pre>';
} catch (PHPParser_Error $e) {
echo 'Parse Error: ', $e->getMessage();
}
```
Produces:
```xml
<?xml version="1.0" encoding="UTF-8"?>
<AST xmlns:node="http://nikic.github.com/PHPParser/XML/node" xmlns:subNode="http://nikic.github.com/PHPParser/XML/subNode" xmlns:scalar="http://nikic.github.com/PHPParser/XML/scalar">
<scalar:array>
<node:Stmt_Function line="2">
<subNode:byRef>
<scalar:false/>
</subNode:byRef>
<subNode:params>
<scalar:array>
<node:Param line="2">
<subNode:name>
<scalar:string>msg</scalar:string>
</subNode:name>
<subNode:default>
<scalar:null/>
</subNode:default>
<subNode:type>
<scalar:null/>
</subNode:type>
<subNode:byRef>
<scalar:false/>
</subNode:byRef>
</node:Param>
</scalar:array>
</subNode:params>
<subNode:stmts>
<scalar:array>
<node:Stmt_Echo line="3">
<subNode:exprs>
<scalar:array>
<node:Expr_Variable line="3">
<subNode:name>
<scalar:string>msg</scalar:string>
</subNode:name>
</node:Expr_Variable>
<node:Scalar_String line="3">
<subNode:value>
<scalar:string>
</scalar:string>
</subNode:value>
</node:Scalar_String>
</scalar:array>
</subNode:exprs>
</node:Stmt_Echo>
</scalar:array>
</subNode:stmts>
<subNode:name>
<scalar:string>printLine</scalar:string>
</subNode:name>
</node:Stmt_Function>
<node:Expr_FuncCall line="6">
<subNode:name>
<node:Name line="6">
<subNode:parts>
<scalar:array>
<scalar:string>printLine</scalar:string>
</scalar:array>
</subNode:parts>
</node:Name>
</subNode:name>
<subNode:args>
<scalar:array>
<node:Arg line="6">
<subNode:value>
<node:Scalar_String line="6">
<subNode:value>
<scalar:string>Hallo World!!!</scalar:string>
</subNode:value>
</node:Scalar_String>
</subNode:value>
<subNode:byRef>
<scalar:false/>
</subNode:byRef>
</node:Arg>
</scalar:array>
</subNode:args>
</node:Expr_FuncCall>
</scalar:array>
</AST>
```

View File

@@ -0,0 +1,265 @@
Code generation
===============
It is also possible to generate code using the parser, by first creating an Abstract Syntax Tree and then using the
pretty printer to convert it to PHP code. To simplify code generation, the project comes with a set of builders for
common structures as well as simple templating support. Both features are described in the following:
Builders
--------
The project provides builders for classes, interfaces, methods, functions, parameters and properties, which
allow creating node trees with a fluid interface, instead of instantiating all nodes manually.
Here is an example:
```php
<?php
$factory = new PHPParser_BuilderFactory;
$node = $factory->class('SomeClass')
->extend('SomeOtherClass')
->implement('A\Few', 'Interfaces')
->makeAbstract() // ->makeFinal()
->addStmt($factory->method('someMethod')
->makeAbstract() // ->makeFinal()
->addParam($factory->param('someParam')->setTypeHint('SomeClass'))
)
->addStmt($factory->method('anotherMethod')
->makeProtected() // ->makePublic() [default], ->makePrivate()
->addParam($factory->param('someParam')->setDefault('test'))
// it is possible to add manually created nodes
->addStmt(new PHPParser_Node_Expr_Print(new PHPParser_Node_Expr_Variable('someParam')))
)
// properties will be correctly reordered above the methods
->addStmt($factory->property('someProperty')->makeProtected())
->addStmt($factory->property('anotherProperty')->makePrivate()->setDefault(array(1, 2, 3)))
->getNode()
;
$stmts = array($node);
echo $prettyPrinter->prettyPrint($stmts);
```
This will produce the following output with the default pretty printer:
```php
<?php
abstract class SomeClass extends SomeOtherClass implements A\Few, Interfaces
{
protected $someProperty;
private $anotherProperty = array(1, 2, 3);
abstract function someMethod(SomeClass $someParam);
protected function anotherMethod($someParam = 'test')
{
print $someParam;
}
}
```
Templates
---------
Additionally it is possible to generate code from reusable templates.
As an example consider the following template, which defines a general getter/setter skeleton in terms of a property
`__name__` and its `__type__`:
```php
<?php
class GetterSetterTemplate
{
/**
* @var __type__ The __name__
*/
protected $__name__;
/**
* Gets the __name__.
*
* @return __type__ The __name__
*/
public function get__Name__() {
return $this->__name__;
}
/**
* Sets the __name__.
*
* @param __type__ $__name__ The new __name__
*/
public function set__Name__($__name__) {
$this->__name__ = $__name__;
}
}
```
Using this template we can easily create a class with multiple properties and their respective getters and setters:
```php
<?php
// $templateString contains the above template
$template = new PHPParser_Template($parser, $templateString);
// We only have to specify the __name__ placeholder, as the
// capitalized __Name__ placeholder is automatically created
$properties = [
['name' => 'title', 'type' => 'string'],
['name' => 'body', 'type' => 'string'],
['name' => 'author', 'type' => 'User'],
['name' => 'timestamp', 'type' => 'DateTime'],
];
$class = $factory->class('BlogPost')->implement('Post');
foreach ($properties as $propertyPlaceholders) {
$stmts = $template->getStmts($propertyPlaceholders);
$class->addStmts(
// $stmts contains all statements from the template. So [0] fetches the class statement
// and ->stmts retrieves the methods.
$stmts[0]->stmts
);
}
echo $prettyPrinter->prettyPrint(array($class->getNode()));
```
The result would look roughly like this:
```php
<?php
class BlogPost implements Post
{
/**
* @var string The title
*/
protected $title;
/**
* @var string The body
*/
protected $body;
/**
* @var User The author
*/
protected $author;
/**
* @var DateTime The timestamp
*/
protected $timestamp;
/**
* Gets the title.
*
* @return string The title
*/
public function getTitle()
{
return $this->title;
}
/**
* Sets the title.
*
* @param string $title The new title
*/
public function setTitle($title)
{
$this->title = $title;
}
/**
* Gets the body.
*
* @return string The body
*/
public function getBody()
{
return $this->body;
}
/**
* Sets the body.
*
* @param string $body The new body
*/
public function setBody($body)
{
$this->body = $body;
}
/**
* Gets the author.
*
* @return User The author
*/
public function getAuthor()
{
return $this->author;
}
/**
* Sets the author.
*
* @param User $author The new author
*/
public function setAuthor($author)
{
$this->author = $author;
}
/**
* Gets the timestamp.
*
* @return DateTime The timestamp
*/
public function getTimestamp()
{
return $this->timestamp;
}
/**
* Sets the timestamp.
*
* @param DateTime $timestamp The new timestamp
*/
public function setTimestamp($timestamp)
{
$this->timestamp = $timestamp;
}
}
```
When using multiple templates it is easier to manage them on the filesystem. They can be loaded using the
`TemplateLoader`:
```php
<?php
// We'll store our templates in ./templates and give them a .php suffix
$loader = new PHPParser_TemplateLoader($parser, './templates', '.php');
// loads ./templates/GetterSetter.php
$getterSetterTemplate = $loader->load('GetterSetter');
// loads ./templates/Collection.php
$collectionTemplate = $loader->load('Collection');
// The use of a suffix is optional. The following code for example is equivalent:
$loader = new PHPParser_TemplateLoader($parser, './templates');
// loads ./templates/GetterSetter.php
$getterSetterTemplate = $loader->load('GetterSetter.php');
// loads ./templates/Collection.php
$collectionTemplate = $loader->load('Collection.php');
```

View File

@@ -0,0 +1,114 @@
Lexer component documentation
=============================
The lexer is responsible for providing tokens to the parser. The project comes with two lexers: `PHPParser_Lexer` and
`PHPParser_Lexer_Emulative`. The latter is an extension of the former, which adds the ability to emulate tokens of
newer PHP versions and thus allows parsing of new code on older versions.
A lexer has to define the following public interface:
startLexing($code);
getNextToken(&$value = null, &$startAttributes = null, &$endAttributes = null);
handleHaltCompiler();
startLexing
-----------
The `startLexing` method is invoked when the `parse()` method of the parser is called. It's argument will be whatever
was passed to the `parse()` method.
Even though `startLexing` is meant to accept a source code string, you could for example overwrite it to accept a file:
```php
<?php
class FileLexer extends PHPParser_Lexer {
public function startLexing($fileName) {
if (!file_exists($fileName)) {
throw new InvalidArgumentException(sprintf('File "%s" does not exist', $fileName));
}
parent::startLexing(file_get_contents($fileName));
}
}
$parser = new PHPParser_Parser(new FileLexer);
var_dump($parser->parse('someFile.php'));
var_dump($parser->parse('someOtherFile.php'));
```
getNextToken
------------
`getNextToken` returns the ID of the next token and sets some additional information in the three variables which it
accepts by-ref. If no more tokens are available it has to return `0`, which is the ID of the `EOF` token.
The first by-ref variable `$value` should contain the textual content of the token. It is what will be available as `$1`
etc in the parser.
The other two by-ref variables `$startAttributes` and `$endAttributes` define which attributes will eventually be
assigned to the generated nodes: The parser will take the `$startAttributes` from the first token which is part of the
node and the `$endAttributes` from the last token that is part of the node.
E.g. if the tokens `T_FUNCTION T_STRING ... '{' ... '}'` constitute a node, then the `$startAttributes` from the
`T_FUNCTION` token will be taken and the `$endAttributes` from the `'}'` token.
By default the lexer creates the attributes `startLine`, `comments` (both part of `$startAttributes`) and `endLine`
(part of `$endAttributes`).
If you don't want all these attributes to be added (to reduce memory usage of the AST) you can simply remove them by
overriding the method:
```php
<?php
class LessAttributesLexer extends PHPParser_Lexer {
public function getNextToken(&$value = null, &$startAttributes = null, &$endAttributes = null) {
$tokenId = parent::getNextToken($value, $startAttributes, $endAttributes);
// only keep startLine attribute
unset($startAttributes['comments']);
unset($endAttributes['endLine']);
return $tokenId;
}
}
```
You can obviously also add additional attributes. E.g. in conjunction with the above `FileLexer` you might want to add
a `fileName` attribute to all nodes:
```php
<?php
class FileLexer extends PHPParser_Lexer {
protected $fileName;
public function startLexing($fileName) {
if (!file_exists($fileName)) {
throw new InvalidArgumentException(sprintf('File "%s" does not exist', $fileName));
}
$this->fileName = $fileName;
parent::startLexing(file_get_contents($fileName));
}
public function getNextToken(&$value = null, &$startAttributes = null, &$endAttributes = null) {
$tokenId = parent::getNextToken($value, $startAttributes, $endAttributes);
// we could use either $startAttributes or $endAttributes here, because the fileName is always the same
// (regardless of whether it is the start or end token). We choose $endAttributes, because it is slightly
// more efficient (as the parser has to keep a stack for the $startAttributes).
$endAttributes['fileName'] = $fileName;
return $tokenId;
}
}
```
handleHaltCompiler
------------------
The method is invoked whenever a `T_HALT_COMPILER` token is encountered. It has to return the remaining string after the
construct (not including `();`).