Bug in FOR command of cmd.exe

  • Thread starter Thread starter kamyers1
  • Start date Start date
K

kamyers1

I wrote a couple of simple utilities using the FOR command in .bat files to
simplify string handling. These utilities 1) remove quotes, and 2) extract
the Nth word. They can both 1) process constant text from the command line,
2) process text output from another command entered on the command line, and
3) act as filters for text piped from another command.

Both of these utilities mostly work as intended. However, there is some
kind of problem that frequently results in printing a generally garbled
version of the message "The system cannot find the file". The message is
garbled in such a way as to strongly suggest that memory is somehow being
trashed.

These errors seem to usually (always?) result from using the FOR command
with the /F option to process contant text that has been enclosed in single
quotes, when the FOR command is NOT even being used to work with files. The
errors seem to occur *after* the FOR command has finished processing and is
terminating or cleaning up after itself.

I would appreciate if someone else could test these scripts to see if they
are able to duplicate the same errors that I see, and also to let me know if
you see anything that I am doing wrong. The content of my two scripts is
provided below, along with a simple test script that frequently produces the
error message. The test script runs in a continuous loop and must be
terminated using Ctrl-C when you are ready to quit.

temp.bat:

@echo off

rem Put this in the same folder as the other two files,
rem and run it from a command promp to test them.
rem Theoretically you should see three lines containing
rem "this is a test", followed by three lines with "is",
rem but you will probably also see some garbled
rem "The system cannot find the file" messages.
rem Hit Ctrl-c to exit when you have seen enough.

:start

echo "this is a test"|cmd /c dequote
call dequote /t "this is a test"
call dequote echo "this is a test"
echo.

echo this is a test|cmd /c scan 2
call scan 2 /t this is a test
call scan 2 echo this is a test
echo.

goto start


dequote.bat:

@echo off


:init

if '%1' equ '/?' goto help
if '%1' equ '-?' goto help
if /I '%1' equ '/h' goto help
if /I '%1' equ '-h' goto help


:main

if '%1' equ '' (
rem deQuote as a filter, could return multiple lines
rem for /f "tokens=* usebackq" %%s in (`NullFilter`) do echo %%~s
for /f "tokens=* usebackq" %%s in (`find /v "xyzzyx963852741"`) do echo %%~s
) else (
if /I '%1' equ '/t' (
rem deQuote text from command line, always a single line
for /f "tokens=1,* usebackq" %%a in ('%*') do echo %%~b
) else (
rem deQuote output from specified command, could return multiple lines
for /f "tokens=* usebackq" %%o in (`%*`) do echo %%~o
)
)

goto end


:help

echo deQuote.bat
echo Copyright (c) 2009-09-11 by Kevin A. Myers
echo All rights reserved.
echo.
echo Removes enclosing quotes from source text.
echo.
echo Syntax:
echo deQuote
echo deQuote /t <text>
echo deQuote <command>
echo.


:end


scan.bat:

@echo off


:init

if '%1' equ '/?' goto help
if '%1' equ '-?' goto help
if /I '%1' equ '/h' goto help
if /I '%1' equ '-h' goto help


:main

setlocal

if '%1' equ '' (set token=1) else (set token=%~1)

if '%delims%' neq '' set delims=delims=%delims%

if '%2' equ '' (
rem scan as a filter
rem for /f "tokens=%token% usebackq %delims%" %%s in (`NullFilter`) do echo
%%s
for /f "tokens=%token% usebackq %delims%" %%s in (`find /v
"xyzzyx963852741"`) do echo %%s
) else (
if /I '%2' equ '/t' (
rem scan text from command line
for /f "tokens=2,* usebackq" %%a in ('%*') do (
for /f "tokens=%token% usebackq %delims%" %%t in ('%%b') do echo %%t
)
) else (
rem scan output from specified command
for /f "tokens=1,* usebackq" %%a in ('%*') do (
for /f "tokens=%token% usebackq %delims%" %%o in (`%%b`) do echo %%o
)
)
)

endlocal

goto end


:help

echo scan.bat
echo Copyright (c) 2009-09-11 by Kevin A. Myers
echo All rights reserved.
echo.
echo Extract token(s) from specified input line(s).
echo Delimiters are taken from a delims environment variable.
echo.
echo Syntax:
echo scan
echo scan <token>
echo scan <token> /t <text>
echo scan <token> <command>
echo.


:end
 
I wrote a couple of simple utilities using the FOR command in .bat files to
simplify string handling.  These utilities 1) remove quotes, and 2) extract
the Nth word.  They can both 1) process constant text from the command line,
2) process text output from another command entered on the command line, and
3) act as filters for text piped from another command.

Both of these utilities mostly work as intended.  However, there is some
kind of problem that frequently results in printing a generally garbled
version of the message "The system cannot find the file".  The message is
garbled in such a way as to strongly suggest that memory is somehow being
trashed.

These errors seem to usually (always?) result from using the FOR command
with the /F option to process contant text that has been enclosed in single
quotes, when the FOR command is NOT even being used to work with files.  The
errors seem to occur *after* the FOR command has finished processing and is
terminating or cleaning up after itself.

I would appreciate if someone else could test these scripts to see if they
are able to duplicate the same errors that I see, and also to let me knowif
you see anything that I am doing wrong.  The content of my two scripts is
provided below, along with a simple test script that frequently produces the
error message.  The test script runs in a continuous loop and must be
terminated using Ctrl-C when you are ready to quit.

temp.bat:

@echo off

rem Put this in the same folder as the other two files,
rem and run it from a command promp to test them.
rem Theoretically you should see three lines containing
rem "this is a test", followed by three lines with "is",
rem but you will probably also see some garbled
rem "The system cannot find the file" messages.
rem Hit Ctrl-c to exit when you have seen enough.

:start

echo "this is a test"|cmd /c dequote
call dequote /t "this is a test"
call dequote echo "this is a test"
echo.

echo this is a test|cmd /c scan 2
call scan 2 /t this is a test
call scan 2 echo this is a test
echo.

goto start

dequote.bat:

@echo off

:init

if '%1' equ '/?' goto help
if '%1' equ '-?' goto help
if /I '%1' equ '/h' goto help
if /I '%1' equ '-h' goto help

:main

if '%1' equ '' (
        rem deQuote as a filter, could return multiple lines
        rem for /f "tokens=* usebackq" %%s in (`NullFilter`) doecho %%~s
        for /f "tokens=* usebackq" %%s in (`find /v "xyzzyx963852741"`) do echo %%~s
) else (
        if /I '%1' equ '/t' (
                rem deQuote text from command line, always a single line
                for /f "tokens=1,* usebackq" %%a in ('%*') do echo %%~b
        ) else (
                rem deQuote output from specified command, could return multiple lines
                for /f "tokens=* usebackq" %%o in (`%*`) do echo %%~o
        )
)

goto end

:help

echo deQuote.bat
echo Copyright (c) 2009-09-11 by Kevin A. Myers
echo All rights reserved.
echo.
echo Removes enclosing quotes from source text.
echo.
echo Syntax:
echo deQuote
echo deQuote /t <text>
echo deQuote <command>
echo.

:end

scan.bat:

@echo off

:init

if '%1' equ '/?' goto help
if '%1' equ '-?' goto help
if /I '%1' equ '/h' goto help
if /I '%1' equ '-h' goto help

:main

setlocal

if '%1' equ '' (set token=1) else (set token=%~1)

if '%delims%' neq '' set delims=delims=%delims%

if '%2' equ '' (
        rem scan as a filter
        rem for /f "tokens=%token% usebackq %delims%" %%s in (`NullFilter`) do echo
%%s
        for /f "tokens=%token% usebackq %delims%" %%s in (`find/v
"xyzzyx963852741"`) do echo %%s
) else (
        if /I '%2' equ '/t' (
                rem scan text from command line
                for /f "tokens=2,* usebackq" %%a in ('%*') do (
                        for /f "tokens=%token% usebackq %delims%" %%t in ('%%b') do echo %%t
                )
        ) else (
                rem scan output from specified command
                for /f "tokens=1,* usebackq" %%a in ('%*') do (
                        for /f "tokens=%token% usebackq %delims%" %%o in (`%%b`) do echo %%o
                )
        )
)

endlocal

goto end

:help

echo scan.bat
echo Copyright (c) 2009-09-11 by Kevin A. Myers
echo All rights reserved.
echo.
echo Extract token(s) from specified input line(s).
echo Delimiters are taken from a delims environment variable.
echo.
echo Syntax:
echo scan
echo scan <token>
echo scan <token> /t <text>
echo scan <token> <command>
echo.

:end
Click to expand...

Since this is far from a trivial set of batch files, you should ask
the batch file experts in alt.msdos.batch.nt for advice. They love
this type of challenge!
 
Since this is far from a trivial set of batch files, you should ask
the batch file experts in alt.msdos.batch.nt for advice. They love
this type of challenge!- Hide quoted text -
Click to expand...

Actually, if you look a little closer you will find those batch files
are really quite simple. They both essentially boil down to executing
some FOR commands that look similar to this:

for /f "tokens=4 usebackq" %%a in ('a b c d e f g h') do echo %%a

According to the docs, that command should produce the command "echo
d", which should in turn echo the d character to the screen.
In fact that does happen. However, quite often, under some
circumstances that I'm not completely sure of, it also produces a
garbled error message *after* the echo command.
The error message is usually "The system cannot find the file "
followed by some graphics characters, but sometimes it is more messed
up than that.
Sometimes I get this error after executing a command similar to the
above only once, other times it takes more tries.
It appears to be affected but not eliminated by use of "set local
enabledelayedexpansion".

Here is a simple test that you can try.
I have tested this on two different machines, both running Windows XP
Service Pack 3 with all critical updates installed.
The results includes garbled error messages on both machines.
Copy the following into a batch file on your machine, then run the
batch file from a command prompt.

@echo off

for /f "tokens=4 usebackq" %%a in ('a b c d e f g h') do echo %%a
for /f "tokens=4 usebackq" %%b in ('a b c d e f g h') do echo %%b
for /f "tokens=4 usebackq" %%c in ('a b c d e f g h') do echo %%c
for /f "tokens=4 usebackq" %%d in ('a b c d e f g h') do echo %%d
for /f "tokens=4 usebackq" %%e in ('a b c d e f g h') do echo %%e
for /f "tokens=4 usebackq" %%f in ('a b c d e f g h') do echo %%f
for /f "tokens=4 usebackq" %%g in ('a b c d e f g h') do echo %%g
for /f "tokens=4 usebackq" %%h in ('a b c d e f g h') do echo %%h
for /f "tokens=4 usebackq" %%i in ('a b c d e f g h') do echo %%i
for /f "tokens=4 usebackq" %%j in ('a b c d e f g h') do echo %%j
for /f "tokens=4 usebackq" %%k in ('a b c d e f g h') do echo %%k
for /f "tokens=4 usebackq" %%l in ('a b c d e f g h') do echo %%l
for /f "tokens=4 usebackq" %%m in ('a b c d e f g h') do echo %%m
for /f "tokens=4 usebackq" %%n in ('a b c d e f g h') do echo %%n
for /f "tokens=4 usebackq" %%o in ('a b c d e f g h') do echo %%o
for /f "tokens=4 usebackq" %%p in ('a b c d e f g h') do echo %%p
 
Back
Top