This is my OLD blog. I've copied this post over to my NEW blog at:
http://www.saltycrane.com/blog/2007/11/remove-c-comments-python/
You should be redirected in 2 seconds.
The Perl FAQ has an
entry How
do I use a regular expression to strip C style comments from a
file? Since I've switched to Python, I've adapted the Perl
solution to Python.
I included two versions from the Perl FAQ.
The simple version removes single-line or multi-line C-style comments
from a file, but has the possibly unwanted behavior of removing
text which looks like comments from within a quoted string. The advantage
is that it uses a regular expression that is easier to understand.
The second version is more robust-- it handles comments within a quoted
string properly (i.e. it does not remove them). This is the recommended version.
Simple version
The simple version uses a regular expression that is only 9 characters long.
The key to the regular expression is the use of the lazy, or non-greedy,
quantifier, *?. From Mastering Regular Expressiongs, Second
Edition by Jeffrey E. F. Friedl:
Quantifiers are normally "greedy", and try to match as much as possible.
Conversely, these non-greedy versions match as little as possible, just the bare
minimum needed to satisfy the match.
Matching a C-style comment using such a simple pattern would not be possible
without the use of lazy quantifiers because of the C-style comment's two
character ending.
remove_comments_simple.py:
# remove_comments_simple.py
import re
import sys
# open file
filename = sys.argv[1]
code_with_comments = open(filename).read()
# strip comments
regex = re.compile(r"/\*.*?\*/", re.MULTILINE|re.DOTALL)
code_without_comments = regex.sub("", code_with_comments)
# write new file
fh = open(filename+".nocomments", "w")
fh.write(code_without_comments)
fh.close()
Example:
To test the script, I created a test file called
testfile.c:
/* This is a C-style comment. */
This is not a comment.
/* This is another
* C-style comment.
*/
"This is /* also not a comment */"
Run the script:
To use the script, I put my script,
remove_comments_simple.py,
and my test file,
testfile.c, in the same directory and ran the
following command:
python remove_comments_simple.py testfile.c
Results:
The script created a new file called
testfile.c.nocomments:
This is not a comment.
"This is "
Robust version (Recommended)
From the Perl FAQ, this version was created by Jeffrey Friedl and
later modified by Fred Curtis. I'm not certain, but this version
appears to use the "unrolling the loop" technique described in
Chapter 6 of Mastering Regular Expressions.
remove_comments.py:
# remove_comments.py
import re
def remove_comments(text):
""" remove c-style comments.
text: blob of text with comments (can include newlines)
returns: text with comments removed
"""
pattern = r"""
## --------- COMMENT ---------
/\* ## Start of /* ... */ comment
[^*]*\*+ ## Non-* followed by 1-or-more *'s
( ##
[^/*][^*]*\*+ ##
)* ## 0-or-more things which don't start with /
## but do end with '*'
/ ## End of /* ... */ comment
| ## -OR- various things which aren't comments:
( ##
## ------ " ... " STRING ------
" ## Start of " ... " string
( ##
\\. ## Escaped char
| ## -OR-
[^"\\] ## Non "\ characters
)* ##
" ## End of " ... " string
| ## -OR-
##
## ------ ' ... ' STRING ------
' ## Start of ' ... ' string
( ##
\\. ## Escaped char
| ## -OR-
[^'\\] ## Non '\ characters
)* ##
' ## End of ' ... ' string
| ## -OR-
##
## ------ ANYTHING ELSE -------
. ## Anything other char
[^/"'\\]* ## Chars which doesn't start a comment, string
) ## or escape
"""
regex = re.compile(pattern, re.VERBOSE|re.MULTILINE|re.DOTALL)
noncomments = [m.group(2) for m in regex.finditer(text) if m.group(2)]
return "".join(noncomments)
if __name__ == '__main__':
filename = sys.argv[1]
code_w_comments = open(filename).read()
code_wo_comments = remove_comments(code_w_comments)
fh = open(filename+".nocomments", "w")
fh.write(code_wo_comments)
fh.close()
Example:
To test this script, I used the same test file,
testfile.c:
/* This is a C-style comment. */
This is not a comment.
/* This is another
* C-style comment.
*/
"This is /* also not a comment */"
Run the script:
To use the script, I put the script,
remove_comments.py,
and my test file,
testfile.c, in the same directory and ran the
following command:
python remove_comments.py testfile.c
Results:
The script created a new file called
testfile.c.nocomments:
This is not a comment.
"This is /* also not a comment */"
---------------
Minor note on Perl to Python migration:
I modified the original regular expression comments a little bit.
In particular, I had to put at least one character after the
##
Non "\ and
## Non '\ lines because, in Python,
the backslash was escaping the following newline character and the
closing parenthesis on the following line was being treated as a
comment by the regular expression engine. This is the error I got,
before the fix:
$ python remove_comments.py
Traceback (most recent call last):
File "remove_comments.py", line 39, in <module>
regex = re.compile(pattern, re.VERBOSE|re.MULTILINE|re.DOTALL)
File "C:\Programs\Python25\lib\re.py", line 180, in compile
return _compile(pattern, flags)
File "C:\Programs\Python25\lib\re.py", line 233, in _compile
raise error, v # invalid expression
sre_constants.error: unbalanced parenthesis
[Read the full post...]
