## FilterChains and FilterReaders

Consider the flexibility of Unix pipes. If you wanted, for example, to copy just those lines that contained the string blee from the first 10 lines of a text file `foo` (you wouldn't want to filter a binary file) to a file `bar`, you would do something like:

    cat foo|head -n10|grep blee > bar

Apache Ant was not flexible enough. There was no way for the `<copy>` task to do something similar. If you wanted the `<copy>` task to get the first 10 lines, you would have had to create special attributes:

    <copy file="foo" tofile="bar" head="10" contains="blee"/>

The obvious problem thus surfaced: Ant tasks would not be able to accommodate such data transformation attributes as they would be endless. The task would also not know in which order these attributes were to be interpreted. That is, must the task execute the contains attribute first and then the head attribute or vice-versa? What Ant tasks needed was a mechanism to allow pluggable filter (data transformer) chains. Ant would provide a few filters for which there have been repeated requests. Users with special filtering needs would be able to easily write their own and plug them in.

The solution was to refactor data transformation oriented tasks to support FilterChains. A FilterChain is a group of ordered FilterReaders. Users can define their own FilterReaders by just extending the `java.io.FilterReader` class. Such custom FilterReaders can be easily plugged in as nested elements of `<filterchain>` by using `<filterreader>` elements.

Example:

    <copy file="${src.file}" tofile="${dest.file}">
      <filterchain>
        <filterreader classname="your.extension.of.java.io.FilterReader">
          <param name="foo" value="bar"/>
        </filterreader>
        <filterreader classname="another.extension.of.java.io.FilterReader">
          <classpath>
            <pathelement path="${classpath}"/>
          </classpath>
          <param name="blah" value="blee"/>
          <param type="abra" value="cadabra"/>
        </filterreader>
      </filterchain>
    </copy>

Ant provides some built-in filter readers. These filter readers can also be declared using a syntax similar to the above syntax. However, they can be declared using some simpler syntax also.

Example:

    <loadfile srcfile="${src.file}" property="src.file.head">
      <filterchain>
        <headfilter lines="15"/>
      </filterchain>
    </loadfile>

is equivalent to:

    <loadfile srcfile="${src.file}" property="src.file.head">
      <filterchain>
        <filterreader classname="org.apache.tools.ant.filters.HeadFilter">
          <param name="lines" value="15"/>
        </filterreader>
      </filterchain>
    </loadfile>

The following built-in tasks support nested `<filterchain>` elements.
Concat,
Copy,
LoadFile,
LoadProperties,
LoadResource,
Move

A FilterChain is formed by defining zero or more of the following nested elements.
FilterReader
ClassConstants
EscapeUnicode
ExpandProperties
HeadFilter
LineContains
LineContainsRegExp
PrefixLines
ReplaceTokens
StripJavaComments
StripLineBreaks
StripLineComments
SuffixLines
TabsToSpaces
TailFilter
DeleteCharacters
ConcatFilter
SortFilter
TokenFilter
FixCRLF

### FilterReader

The `filterreader` element is the generic way to define a filter. User defined filter elements are defined in the build file using this. Please note that built in filter readers can also be defined using this syntax.

A FilterReader element must be supplied with a class name as an attribute value. The class resolved by this name must extend `java.io.FilterReader`. If the custom filter reader needs to be parameterized, it must implement `org.apache.tools.type.Parameterizable`.

Attribute | Description | Required
--- | --- | ---
classname | The class name of the filter reader. | Yes

#### Nested elements

`<filterreader>` supports `<classpath>` and `<param>` as nested elements. Each `<param>` element may take in the following attributes—name, type and value.

The following FilterReaders are supplied with the default distribution.

### ClassConstants

This filters basic constants defined in a Java Class, and outputs them in lines composed of the format name=value. This filter uses the BCEL library to understand the Java Class file. See Library Dependencies.

Important: This filter is different from most of the other filters. Most of the filters operate on a sequence of characters. This filter operates on the sequence of bytes that makes up a class. However the bytes arrive to the filter as a sequence of characters. This means that one must be careful on the choice of character encoding to use. Most encodings lose information on conversion from an arbitrary sequence of bytes to characters and back again to bytes. In particular, the usual default character encodings (CP1252 and UTF-8) do. For this reason, since Ant 1.7, the character encoding ISO-8859-1 is used to convert from characters back to bytes, so one has to use this encoding for reading the Java class file.

#### Example

This loads the basic constants defined in a Java class as Ant properties.

    <loadproperties srcfile="foo.class" encoding="ISO-8859-1">
      <filterchain>
        <classconstants/>
      </filterchain>
    </loadproperties>

This loads the constants from a Java class file as Ant properties, prepending the names with a prefix.

    <loadproperties srcfile="build/classes/org/acme/bar.class"
                    encoding="ISO-8859-1">
      <filterchain>
        <classconstants/>
        <prefixlines prefix="ini."/>
      </filterchain>
    </loadproperties>

### EscapeUnicode

This filter converts its input by changing all non US-ASCII characters into their equivalent Unicode escape backslash u plus 4 digits.

Since Ant 1.6

#### Example

This loads the basic constants defined in a Java class as Ant properties.

    <loadproperties srcfile="non_ascii_property.properties">
      <filterchain>
        <filterreader classname="org.apache.tools.ant.filters.EscapeUnicode"/>
      </filterchain>
    </loadproperties>

Convenience method:

    <loadproperties srcfile="non_ascii_property.properties">
      <filterchain>
        <escapeunicode/>
      </filterchain>
    </loadproperties>

### ExpandProperties

If the data contains data that represents Ant properties (of the form `${...}`), that is substituted with the property's actual value.

#### Example

This results in the property `modifiedmessage` holding the value All these moments will be lost in time, like teardrops in the rain

    <echo message="All these moments will be lost in time, like teardrops in the ${weather}"
          file="loadfile1.tmp"/>
    <property name="weather" value="rain"/>
    <loadfile property="modifiedmessage" srcFile="loadfile1.tmp">
      <filterchain>
        <filterreader classname="org.apache.tools.ant.filters.ExpandProperties"/>
      </filterchain>
    </loadfile>

Convenience method:

    <echo message="All these moments will be lost in time, like teardrops in the ${weather}"
          file="loadfile1.tmp"/>
    <property name="weather" value="rain"/>
    <loadfile property="modifiedmessage" srcFile="loadfile1.tmp">
      <filterchain>
        <expandproperties/>
      </filterchain>
    </loadfile>

Since Ant 1.8.3, a nested PropertySet can be specified:

    <property name="weather" value="rain"/>
    <loadfile property="modifiedmessage" srcFile="loadfile1.tmp">
      <filterchain>
        <expandproperties>
          <propertyset>
            <propertyref name="weather"/>
          </propertyset>
        </expandproperties>
      </filterchain>
    </loadfile>

### HeadFilter

This filter reads the first few lines from the data supplied to it.

Parameter Name | Parameter Value | Required
--- | --- | ---
lines | Number of lines to be read. A negative value means that all lines are passed (useful with skip). | No; defaults to 10
skip | Number of lines to be skipped (from the beginning). | No; defaults to 0

#### Example

This stores the first 15 lines of the supplied data in the property `src.file.head`

    <loadfile srcfile="${src.file}" property="src.file.head">
      <filterchain>
        <filterreader classname="org.apache.tools.ant.filters.HeadFilter">
          <param name="lines" value="15"/>
        </filterreader>
      </filterchain>
    </loadfile>

Convenience method:

    <loadfile srcfile="${src.file}" property="src.file.head">
      <filterchain>
        <headfilter lines="15"/>
      </filterchain>
    </loadfile>

This stores the first 15 lines, skipping the first 2 lines, of the supplied data in the property `src.file.head` (means: lines 3-17).

    <loadfile srcfile="${src.file}" property="src.file.head">
      <filterchain>
        <headfilter lines="15" skip="2"/>
      </filterchain>
    </loadfile>

See the testcases for more examples (`src\etc\testcases\filters\head-tail.xml` in the source distribution).

### LineContains

This filter includes only those lines that contain all the user-specified strings.

Parameter Type | Parameter Value | Required
--- | --- | ---
contains | Substring to be searched for. | Yes
negate | Whether to select non-matching lines only. Since Ant 1.7 | No
matchAny | If false, then all the strings are expected to be present in the line. If true, then the presence of any of the strings in the line is considered a successful match. Since Ant 1.10.4 | No; defaults to false

#### Example

This will include only those lines that contain foo and bar.

    <filterreader classname="org.apache.tools.ant.filters.LineContains">
      <param type="contains" value="foo"/>
      <param type="contains" value="bar"/>
    </filterreader>

Convenience method:

    <linecontains>
      <contains value="foo"/>
      <contains value="bar"/>
    </linecontains>

Negation:

    <filterreader classname="org.apache.tools.ant.filters.LineContains">
      <param type="negate" value="true"/>
      <param type="contains" value="foo"/>
      <param type="contains" value="bar"/>
    </filterreader>

or

    <linecontains negate="true">
      <contains value="foo"/>
      <contains value="bar"/>
    </linecontains>

### LineContainsRegExp

Filter which includes only those lines that contain the user-specified regular expression matching strings.

Parameter Type | Parameter Value | Required
--- | --- | ---
regexp | Regular expression to be searched for. Since Ant 1.10.2, this also works as an attribute on `linecontainsregexp`. In earlier versions of Ant you must use a nested element when using the convenience method. | Yes
negate | Whether to select non-matching lines only. Since Ant 1.7 | No
casesensitive | Perform a case sensitive match. Since Ant 1.8.2 | No; default is true

See Regexp Type for the description of the nested element regexp and of the choice of regular expression implementation.

#### Example

This will fetch all those lines that contain the pattern `foo`

    <filterreader classname="org.apache.tools.ant.filters.LineContainsRegExp">
      <param type="regexp" value="foo*"/>
    </filterreader>

Convenience method:

    <linecontainsregexp>
      <regexp pattern="foo*"/>
    </linecontainsregexp>

Negation:

    <filterreader classname="org.apache.tools.ant.filters.LineContainsRegExp">
      <param type="negate" value="true"/>
      <param type="regexp" value="foo*"/>
    </filterreader>

or

    <linecontainsregexp negate="true">
      <regexp pattern="foo*"/>
    </linecontainsregexp>

### PrefixLines

Attaches a prefix to every line.

Parameter Name | Parameter Value | Required
--- | --- | ---
prefix | Prefix to be attached to lines. | Yes

#### Example

This will attach the prefix `Foo` to all lines.

    <filterreader classname="org.apache.tools.ant.filters.PrefixLines">
      <param name="prefix" value="Foo"/>
    </filterreader>

Convenience method:

    <prefixlines prefix="Foo"/>

### SuffixLines

Since Ant 1.8.0

Attaches a suffix to every line.

Parameter Name | Parameter Value | Required
--- | --- | ---
suffix | Suffix to be attached to lines. | Yes

#### Example

This will attach the suffix `Foo` to all lines.

    <filterreader classname="org.apache.tools.ant.filters.SuffixLines">
      <param name="suffix" value="Foo"/>
    </filterreader>

Convenience method:

    <suffixlines suffix="Foo"/>

### ReplaceTokens

This filter reader replaces all strings that are sandwiched between begintoken and endtoken with user defined values.

Parameter Type | Parameter Name | Parameter Value | Required
--- | --- | --- | ---
tokenchar | begintoken | String marking the beginning of a token. | No; defaults to @
tokenchar | endtoken | String marking the end of a token. | No; defaults to @
token | User defined token string | Replacement value for the token. | Yes
propertiesfile | Not applicable | Properties file to take tokens and replacement values from. | No
propertiesResource | Properties resource to take tokens from. Note this only works is you use the "convenience" `<replacetokens>` syntax. since Ant 1.8.0 | No

#### Example

This replaces occurrences of the string `@DATE@` in the data with today's date and stores it in the property `${src.file.replaced}`.

    <tstamp/>
    <!-- just for explaining the use of the properties -->
    <property name="src.file" value="orders.csv"/>
    <property name="src.file.replaced" value="orders.replaced"/>

    <!-- do the loading and filtering -->
    <loadfile srcfile="${src.file}" property="${src.file.replaced}">
      <filterchain>
        <filterreader classname="org.apache.tools.ant.filters.ReplaceTokens">
          <param type="token" name="DATE" value="${TODAY}"/>
        </filterreader>
      </filterchain>
    </loadfile>

    <!-- just for explaining the use of the properties -->
    <echo message="${orders.replaced}"/>

Convenience method:

    <tstamp/>
    <loadfile srcfile="${src.file}" property="${src.file.replaced}">
      <filterchain>
        <replacetokens>
          <token key="DATE" value="${TODAY}"/>
        </replacetokens>
      </filterchain>
    </loadfile>

This replaces occurrences of the string {{DATE}} in the data with today's date and stores it in the property `${src.file.replaced}`.

    <loadfile srcfile="${src.file}" property="${src.file.replaced}">
      <filterchain>
        <filterreader classname="org.apache.tools.ant.filters.ReplaceTokens">
          <param type="tokenchar" name="begintoken" value="{{"/>
          <param type="tokenchar" name="endtoken" value="}}"/>
        </filterreader>
      </filterchain>
    </loadfile>

Convenience method:

    <tstamp/>
    <loadfile srcfile="${src.file}" property="${src.file.replaced}">
      <filterchain>
        <replacetokens begintoken="{{" endtoken="}}">
          <token key="DATE" value="${TODAY}"/>
        </replacetokens>
      </filterchain>
    </loadfile>

This will treat each properties file entry in sample.properties as a token/key pair:

    <loadfile srcfile="${src.file}" property="${src.file.replaced}">
      <filterchain>
        <filterreader classname="org.apache.tools.ant.filters.ReplaceTokens">
          <param type="propertiesfile" value="sample.properties"/>
        </filterreader>
      </filterchain>
    </loadfile>

This reads the properties from an Ant resource referenced by its id:

    <string id="embedded-properties">
    foo=bar
    baz=xyzzy
    </string>
    <loadfile srcfile="${src.file}" property="${src.file.replaced}">
      <filterchain>
        <replacetokens propertiesResource="${ant.refid:embedded-properties}"/>
      </filterchain>
    </loadfile>

### StripJavaComments

This filter reader strips away comments from the data, using Java syntax guidelines. This filter does not take in any parameters.

#### Example

    <loadfile srcfile="${java.src.file}" property="${java.src.file.nocomments}">
      <filterchain>
        <filterreader classname="org.apache.tools.ant.filters.StripJavaComments"/>
      </filterchain>
    </loadfile>

Convenience method:

    <loadfile srcfile="${java.src.file}" property="${java.src.file.nocomments}">
      <filterchain>
        <stripjavacomments/>
      </filterchain>
    </loadfile>

### StripLineBreaks

This filter reader strips away specific characters from the data supplied to it.

Parameter Name | Parameter Value | Required
--- | --- | ---
linebreaks | Characters that are to be stripped out. | No; defaults to \r\n

#### Examples

This strips the \r and \n characters.

    <loadfile srcfile="${src.file}" property="${src.file.contents}">
      <filterchain>
        <filterreader classname="org.apache.tools.ant.filters.StripLineBreaks"/>
      </filterchain>
    </loadfile>

Convenience method:

    <loadfile srcfile="${src.file}" property="${src.file.contents}">
      <filterchain>
        <striplinebreaks/>
      </filterchain>
    </loadfile>

This treats the ( and ) characters as line break characters and strips them.

    <loadfile srcfile="${src.file}" property="${src.file.contents}">
      <filterchain>
        <filterreader classname="org.apache.tools.ant.filters.StripLineBreaks">
          <param name="linebreaks" value="()"/>
        </filterreader>
      </filterchain>
    </loadfile>

### StripLineComments

This filter removes all those lines that begin with strings that represent comments as specified by the user.

Parameter Type | Parameter Value | Required
--- | --- | ---
comment | Strings that identify a line as a comment when they appear at the start of the line. | Yes

#### Examples

This removes all lines that begin with #, --, REM, rem and //

    <filterreader classname="org.apache.tools.ant.filters.StripLineComments">
      <param type="comment" value="#"/>
      <param type="comment" value="--"/>
      <param type="comment" value="REM "/>
      <param type="comment" value="rem "/>
      <param type="comment" value="//"/>
    </filterreader>

Convenience method:

    <striplinecomments>
      <comment value="#"/>
      <comment value="--"/>
      <comment value="REM "/>
      <comment value="rem "/>
      <comment value="//"/>
    </striplinecomments>

### TabsToSpaces

This filter replaces tabs with spaces

Parameter Name | Parameter Value | Required
--- | --- | ---
tablength | Amount of space characters to replace a tab character with. | No; defaults to 8

#### Examples

This replaces tabs in `${src.file}` with spaces.

    <loadfile srcfile="${src.file}" property="${src.file.notab}">
      <filterchain>
        <filterreader classname="org.apache.tools.ant.filters.TabsToSpaces"/>
      </filterchain>
    </loadfile>

Convenience method:

    <loadfile srcfile="${src.file}" property="${src.file.notab}">
      <filterchain>
        <tabstospaces/>
      </filterchain>
    </loadfile>

### TailFilter

This filter reads the last few lines from the data supplied to it.

Parameter Name | Parameter Value | Required
--- | --- | ---
lines | Number of lines to be read. A negative value means that all lines are passed (useful with skip) | No; defaults to 10
skip | Number of lines to be skipped (from the end). | No; defaults to 0

#### Background

With HeadFilter and TailFilter you can extract each part of a text file you want. This graphic shows the dependencies:

Content | Filter
--- | ---

Line 1

    <filterchain>
        <headfilter lines="2"/>
    </filterchain>

    <filterchain>
        <tailfilter lines="-1" skip="2"/>
    </filterchain>

Line 2

Line 3

    <filterchain>
        <headfilter lines="-1" skip="2"/>
    </filterchain>

    <filterchain>
        <headfilter lines="-1" skip="2"/>
        <tailfilter lines="-1" skip="2"/>
    </filterchain>

Line 4
Line 5
Lines ...
Line 95
Line 96
Line 97

Line 98

    <filterchain>
        <tailfilter lines="2"/>
    </filterchain>

Line 99

#### Examples

This stores the last 15 lines of the supplied data in the property `${src.file.tail}`

    <loadfile srcfile="${src.file}" property="${src.file.tail}">
      <filterchain>
        <filterreader classname="org.apache.tools.ant.filters.TailFilter">
          <param name="lines" value="15"/>
        </filterreader>
      </filterchain>
    </loadfile>

Convenience method:

    <loadfile srcfile="${src.file}" property="${src.file.tail}">
      <filterchain>
        <tailfilter lines="15"/>
      </filterchain>
    </loadfile>

This stores the last 5 lines of the first 15 lines of the supplied data in the property `${src.file.mid}`

    <loadfile srcfile="${src.file}" property="${src.file.mid}">
      <filterchain>
        <filterreader classname="org.apache.tools.ant.filters.HeadFilter">
          <param name="lines" value="15"/>
        </filterreader>
        <filterreader classname="org.apache.tools.ant.filters.TailFilter">
          <param name="lines" value="5"/>
        </filterreader>
      </filterchain>
    </loadfile>

Convenience method:

    <loadfile srcfile="${src.file}" property="${src.file.mid}">
      <filterchain>
        <headfilter lines="15"/>
        <tailfilter lines="5"/>
      </filterchain>
    </loadfile>

This stores the last 10 lines, skipping the last 2 lines, of the supplied data in the property `src.file.head`. (Means: if supplied data contains 60 lines, lines 49-58 are extracted)

    <loadfile srcfile="${src.file}" property="src.file.head">
      <filterchain>
        <tailfilter lines="10" skip="2"/>
      </filterchain>
    </loadfile>

### DeleteCharacters

Since Ant 1.6

This filter deletes specified characters.

This filter is only available in the convenience form.

Parameter Name | Parameter Value | Required
--- | --- | ---
chars | The characters to delete. This attribute is backslash enabled. | Yes

#### Examples

Delete tabs and returns from the data.

    <deletecharacters chars="\t\r"/>

### ConcatFilter

Since Ant 1.6

This filter prepends or appends the content file to the filtered files.

Parameter Name | Parameter Value | Required
--- | --- | ---
prepend | The name of the file which content should be prepended to the file. | No
append | The name of the file which content should be appended to the file. | No

#### Examples

Do nothing:

    <filterchain>
        <concatfilter/>
    </filterchain>

Adds a license text before each Java source file:

    <filterchain>
        <concatfilter prepend="apache-license-java.txt"/>
    </filterchain>

### SortFilter

Since Ant 1.8.0

The sort filter reads all lines and sorts them. The sort order can be reversed and it is possible to specify a custom implementation of the `java.util.Comparator` interface to get even more control.

Parameter Name | Parameter Value | Required
--- | --- | ---
reverse | Whether to reverse the sort order. Note: this parameter is ignored if the comparator parameter is present as well. | No; defaults to false
comparator | Class name of a class that implements `java.util.Comparator` for Strings. This class will be used to determine the sort order of lines. | No

This filter is also available using the name `sortfilter`. The `reverse` parameter becomes an attribute, `comparator` can be specified by using a nested element.

#### Examples

    <copy todir="build">
        <fileset dir="input" includes="*.txt"/>
        <filterchain>
            <sortfilter/>
        </filterchain>
    </copy>

Sort all files `*.txt` from `src` location into `build` location. The lines of each file are sorted in ascendant order comparing the lines via the `String.compareTo(Object o)` method.

    <copy todir="build">
        <fileset dir="input" includes="*.txt"/>
        <filterchain>
            <sortfilter reverse="true"/>
        </filterchain>
    </copy>

Sort all files `*.txt` from `src` location into reverse order and copy them into `build` location.

    <copy todir="build">
        <fileset dir="input" includes="*.txt"/>
        <filterchain>
            <filterreader classname="org.apache.tools.ant.filters.SortFilter">
                <param name="comparator" value="org.apache.tools.ant.filters.EvenFirstCmp"/>
            </filterreader>
        </filterchain>
    </copy>

Sort all files `*.txt` from `src` location using as sorting criterium `EvenFirstCmp` class, that sorts the file lines putting even lines first then odd lines for example. The modified files are copied into `build` location. The `EvenFirstCmp` has to an instanciable class via `Class.newInstance()`, therefore in case of inner class has to be static. It also has to implement `java.util.Comparator` interface, for example:

    package org.apache.tools.ant.filters;
    ... (omitted)
    public final class EvenFirstCmp implements <b>Comparator</b> {
       public int compare(Object o1, Object o2) {
           ...(omitted)
       }
    }

The example above is equivalent to:

    <componentdef name="evenfirst"
                  classname="org.apache.tools.ant.filters.EvenFirstCmp"/>
    <copy todir="build">
        <fileset dir="input" includes="*.txt"/>
        <filterchain>
            <sortfilter>
                <evenfirst/>
            </sortfilter>
        </filterchain>
    </copy>

### TokenFilter

This filter tokenizes the InputStream into strings and passes these strings to filters of strings. Unlike the other FilterReaders, this does not support `param`s, only convenience methods are implemented. The tokenizer and the string filters are defined by nested elements.

Since Ant 1.6

Only one `tokenizer` element may be used, the LineTokenizer is the default if none are specified. A tokenizer splits the input into token strings and trailing delimiter strings.

There may be zero or more string filters. A string filter processes a token and either returns a string or a null. It the string is not null it is passed to the next filter. This proceeds until all the filters are called. If a string is returned after all the filters, the string is outputs with its associated token delimiter (if one is present). The trailing delimiter may be overridden by the delimOutput attribute.

#### Backslash interpretation

A number of attributes (including delimOutput) interpret backslash escapes. The following are understood: \n, \r, \f, \t and \\.

Attribute | Description | Required
--- | --- | ---
delimOutput | This overrides the token delimiter returned by the tokenizer if it is not empty. This attribute is backslash enabled. | No

The following tokenizers are provided by the default distribution.

LineTokenizer
FileTokenizer
StringTokenizer

The following string filters are provided by the default distribution.

ReplaceString
ContainsString
ReplaceRegex
ContainsRegex
Trim
IgnoreBlank
DeleteCharacters
UniqFilter
Native2AsciiFilter

The following string filters are provided by the optional distribution.

ScriptFilter

Custom tokenizers and string filters can be declared using typedef task.

Some of the filters may be used directly within a `filterchain`. In this case a `tokenfilter` is created implicitly. An extra attribute byline is added to the filter to specify whether to use a `linetokenizer` (byline=true) or a `filetokenizer` (byline=false). The default is true.

#### LineTokenizer

This tokenizer splits the input into lines. The tokenizer delimits lines by \r, \n or \r\n. This is the default tokenizer.

Attribute | Description | Required
--- | --- | ---
includeDelims | Include the line endings in the token. | No; default is false

##### Examples

Convert input current line endings to Unix style line endings.

    <tokenfilter delimoutput="\n"/>

Remove blank lines.

    <tokenfilter>
        <ignoreblank/>
    </tokenfilter>

#### FileTokenizer

This tokenizer treats all the input as a token. So be careful not to use this on very large input.

##### Examples

Replace the first occurrence of `package` with `//package`.

    <tokenfilter>
          <filetokenizer/>
          <replaceregex pattern="([\n\r]+[ \t]*|^[ \t]*)package"
                        flags="s"
                        replace="\1//package"/>
    </tokenfilter>

#### StringTokenizer

This tokenizer is based on `java.util.StringTokenizer`. It splits up the input into strings separated by white space, or by a specified list of delimiting characters. If the stream starts with delimiter characters, the first token will be the empty string (unless the delimsaretokens attribute is used).

Attribute | Description | Required
--- | --- | ---
delims | The delimiter characters. | No; defaults to white space as defined by `java.lang.Character.isWhitespace()`
delimsaretokens | If this is true, each delimiter character is returned as a token. | No; default is false
suppressdelims | If this is true, delimiters are not returned. | No; default is false
includeDelims | Include the delimiters in the token. | No; default is false

##### Examples

Surround each non space token with a [].

    <tokenfilter>
        <stringtokenizer/>
        <replaceregex pattern="(.+)" replace="[\1]"/>
    </tokenfilter>

#### ReplaceString

This is a simple filter to replace strings. This filter may be used directly within a `filterchain`.

Attribute | Description | Required
--- | --- | ---
from | The string that must be replaced. | Yes
to | The new value for the replaced string. | No; defaults to empty string

##### Examples

Replace sun with moon.

    <tokenfilter>
        <replacestring from="sun" to="moon"/>
    </tokenfilter>

#### ContainsString

This is a simple filter to filter tokens that contains a specified string.

Attribute | Description | Required
--- | --- | ---
contains | The string that the token must contain. | Yes

##### Examples

Include only lines that contain foo

    <tokenfilter>
        <containsstring contains="foo"/>
    </tokenfilter>

#### ReplaceRegex

This string filter replaces regular expressions. This filter may be used directly within a `filterchain`.

See Regexp Type concerning the choice of the implementation.

Attribute | Description | Required
--- | --- | ---
pattern | The regular expression pattern to match in the token. | Yes
replace | The substitution pattern to replace the matched regular expression. | No; defaults to empty string
flags | See ReplaceRegexp for an explanation of regex flags. | No

##### Examples

Replace all occurrences of hello with world, ignoring case.

    <tokenfilter>
        <replaceregex pattern="hello" replace="world" flags="gi"/>
    </tokenfilter>

#### ContainsRegex

This filters strings that match regular expressions. The filter may optionally replace the matched regular expression. This filter may be used directly within a `filterchain`.

See Regexp Type concerning the choice of regular expression implementation.

Attribute | Description | Required
--- | --- | ---
pattern | The regular expression pattern to match in the token. | Yes
replace | The substitution pattern to replace the matched regular expression. | No; defaults to returning the original token
flags | See ReplaceRegexp for an explanation of regex flags. | No

##### Examples

Filter lines that contain hello or world, ignoring case.

    <tokenfilter>
        <containsregex pattern="(hello|world)" flags="i"/>
    </tokenfilter>

This example replaces lines like `SUITE(TestSuite, bits);` with `void register_bits();` and removes other lines.

    <tokenfilter>
        <containsregex
            pattern="^ *SUITE\(.*,\s*(.*)\s*\).*"
            replace="void register_\1();"/>
    </tokenfilter>

#### Trim

This filter trims whitespace from the start and end of tokens. This filter may be used directly within a `filterchain`.

#### IgnoreBlank

This filter removes empty tokens. This filter may be used directly within a `filterchain`.

#### DeleteCharacters

This filter deletes specified characters from tokens.

Attribute | Description | Required
--- | --- | ---
chars | The characters to delete. This attribute is backslash enabled. | Yes

##### Examples

Delete tabs from lines, trim the lines and removes empty lines.

    <tokenfilter>
        <deletecharacters chars="\t"/>
        <trim/>
        <ignoreblank/>
    </tokenfilter>

#### UniqFilter

Suppresses all tokens that match their ancestor token. It is most useful if combined with a sort filter.

This filter may be used directly within a `filterchain`.

##### Example

This suppresses duplicate lines.

    <tokenfilter>
      <uniqfilter/>
    </tokenfilter>

#### Native2AsciiFilter

Since Ant 1.9.8

Uses the "builtin" implementation of the native2ascii task.

Replaces non-ASCII characters by their Unicode-escapes or vice-versa.

This filter may be used directly within a `filterchain`.

Attribute | Description | Required
--- | --- | ---
reverse | Reverse the sense of the conversion, i.e. convert from ASCII to native. | No

##### Example

This replaces all non-ASCII characters by their Unicode-escapes.

    <tokenfilter>
      <native2asciifilter/>
    </tokenfilter>

#### ScriptFilter

This is an optional filter that executes a script in a Apache BSF or JSR 223 supported language.

See the Script task for an explanation of scripts and dependencies.

The script is provided with an object `self` that has `getToken()` and `setToken(String)` methods. The `getToken()` method returns the current token. The `setToken(String)` method replaces the current token.

This filter may be used directly within a `filterchain`.

Attribute | Description | Required
--- | --- | ---
language | Programming language the script is written in. Must be a supported Apache BSF or JSR 223 language | Yes
manager | Script engine manager to use. See the script task for using this attribute. | No; default is auto
src | Location of the script as a file, if not inline. | No
encoding | Encoding of the script as a file. Since Ant 1.10.2 | No; defaults to default JVM character encoding
setbeans | Whether to have all properties, references and targets as global variables in the script. Since Ant 1.8.0 | No; default is true
classpath | The classpath to pass into the script. | No
classpathref | The classpath to use, given as a reference to a path defined elsewhere. | No

This filter can take a nested `<classpath>` element. See the script task on how to use this element.

##### Examples

Convert to uppercase:

    <tokenfilter>
        <scriptfilter language="javascript">
            self.setToken(self.getToken().toUpperCase());
        </scriptfilter>
    </tokenfilter>

Remove lines containing the string bad while copying text files:

    <copy todir="dist">
      <fileset dir="src" includes="**/*.txt"/>
      <filterchain>
        <scriptfilter language="beanshell">
            if (self.getToken().indexOf("bad") != -1) {
                self.setToken(null);
            }
        </scriptfilter>
      </filterchain>
    </copy>

#### Custom tokenizers and string filters

Custom string filters and tokenizers may be plugged in by extending the interfaces `org.apache.tools.ant.filters.TokenFilter.Filter` and `org.apache.tools.ant.util.Tokenizer` respectly.

They are defined in the build file using `<typedef/>`. For example, a string filter that capitalizes words may be declared as:

    package my.customant;
    import org.apache.tools.ant.filters.TokenFilter;

    public class Capitalize
        implements TokenFilter.Filter
    {
        public String filter(String token) {
            if (token.length() == 0)
                return token;
            return token.substring(0, 1).toUpperCase() +
                    token.substring(1);
       }
    }

This may be used as follows:

    <typedef name="capitalize" classname="my.customant.Capitalize"
             classpath="my.customant.path"/>
    <copy file="input" tofile="output">
      <filterchain>
        <tokenfilter>
          <stringtokenizer/>
          <capitalize/>
        </tokenfilter>
      </filterchain>
    </copy>