Tokenizer
在线手册：中文 英文
PHP手册

范例

Here is a simple example PHP scripts using the tokenizer that will read in a PHP file, strip all comments from the source and print the pure code only.

Example #1 Strip comments with the tokenizer

<?php
/*
* T_ML_COMMENT does not exist in PHP 5.
* The following three lines define it in order to
* preserve backwards compatibility.
*
* The next two lines define the PHP 5 only T_DOC_COMMENT,
* which we will mask as T_ML_COMMENT for PHP 4.
*/
if (!defined('T_ML_COMMENT')) {
   define('T_ML_COMMENT', T_COMMENT);
} else {
   define('T_DOC_COMMENT', T_ML_COMMENT);
}

$source = file_get_contents('example.php');
$tokens = token_get_all($source);

foreach ($tokens as $token) {
   if (is_string($token)) {
       // simple 1-character token
       echo $token;
   } else {
       // token array
       list($id, $text) = $token;

       switch ($id) { 
           case T_COMMENT: 
           case T_ML_COMMENT: // we've defined this
           case T_DOC_COMMENT: // and this
               // no action on comments
               break;

           default:
               // anything else -> output "as is"
               echo $text;
               break;
       }
   }
}
?>
Tokenizer
在线手册：中文 英文
PHP手册
PHP手册 - N: 范例

用户评论:

support at image-host-script dot com (21-Jul-2009 08:15)

Code snippet posted above is perfect enough, and I just wanted to put same code in a function that gets argument as code and returns the comments stripped code, so that it is easy for a beginner too, to copy and use this code.

<?
    if (!defined('T_ML_COMMENT')) {
       define('T_ML_COMMENT', T_COMMENT);
    } else {
       define('T_DOC_COMMENT', T_ML_COMMENT);
    }
    function strip_comments($source) {
        $tokens = token_get_all($source);
        $ret = "";
        foreach ($tokens as $token) {
           if (is_string($token)) {
              $ret.= $token;
           } else {
              list($id, $text) = $token;

              switch ($id) {
                 case T_COMMENT:
                 case T_ML_COMMENT: // we've defined this
                 case T_DOC_COMMENT: // and this
                    break;

                 default:
                    $ret.= $text;
                    break;
              }
           }
        }   
        return trim(str_replace(array('<?','?>'),array('',''),$ret));
    }
?>

1.Now using this function 'strip_comments' for passing code contained in some variable:

<?
$code = "
<?php
    /* this is comment */
   // this is also a comment
   # me too, am also comment
   echo "And I am some code...";
?>";

   $code = strip_comments($code);

   echo htmlspecialchars($code);
?>

Will result output as
<?
   echo "And I am some code...";
?>

2.Loading from a php file:

<?
   $code = file_get_contents("some_code_file.php");
   $code = strip_comments($code);

   echo htmlspecialchars($code);
?>

3. Loading a php file, stripping comments and saving it back

<?
   $file = "some_code_file.php"
   $code = file_get_contents($file);
   $code = strip_comments($code);

   $f = fopen($file,"w");
   fwrite($f,$code);
   fclose($f);
?>

regards
Ali Imran