Tokenizer Component

The Thruster Tokenizer Component.

Install

Via Composer

$ composer require thruster/tokenizer

Usage

use Thruster\Component\Tokenizer\Parser;

$parser = new Parser();

$parser->parse('<?php ... ?>'); // => Token[]
$parser->parseBlock('function fooBar(){...};'); // => Token[]
$parser->parseFile('/some/file.php'); // => Token[]

Testing

$ composer test

Contributing

Please see CONTRIBUTING and CONDUCT for details.

License

Please see License File for more information.