A Message Board, Guestbook, or Poll hosted for your website.
 
Register Login New Posts Chat
 
unmarked software > Forums > General > Possible to tokenize substrings?
 
Username:  
Password:  
 
   
 


Thread Tools Search This Thread 
Reply
 
Author Comment
 
Orionizer
Avatar / Picture

Registered: 06/16/08
Posts: 2

    06/16/08 at 03:01 AM
  Reply with quote#1

I was wondering if it is possible to tokenize substrings based on a delimiter(s).

If so, is it possible to align the tokens as well?

Ex. Let's say I have the following text file:

Smith 2/1/2008 5870.23
Johnson 1/5/2008 340.67
Wilson 10/2/2007 12.44
Johannesberg 5/23/2008 12567.08

I would like the output to look something like:

Smith 2/1/2008 5870.23
Johnson 1/5/2008 340.67
Wilson 10/2/2007 12.44
Johannesberg 5/23/2008 12567.08

Is this possible?
__________________
<a href="http://www.linkedin.com/in/robertdsmith" ><img src="http://www.linkedin.com/img/webpromo/btn_viewmy_160x33.gif" width="160" height="33" border="0" alt="View Robert D. Smith's profile on LinkedIn"></a>
Orionizer
Avatar / Picture

Registered: 06/16/08
Posts: 2

    06/16/08 at 03:02 AM
  Reply with quote#2

Also, please note that this doesn't look correct on the forums - please copy and paste into a text editor with a fixed font to view the examples...
__________________
<a href="http://www.linkedin.com/in/robertdsmith" ><img src="http://www.linkedin.com/img/webpromo/btn_viewmy_160x33.gif" width="160" height="33" border="0" alt="View Robert D. Smith's profile on LinkedIn"></a>
unmarked
Avatar / Picture

Moderator
Registered: 01/15/07
Posts: 278

    06/16/08 at 10:26 AM
  Reply with quote#3

I'm a little confused on what you mean by tokenizing in this context. I'll need to understand the context a little better for a more precise answer.

Given that, I'll take a shot: it looks like you are trying to put tabs between the items in this example. There are a couple approaches possible for this:

1. Use the find multiple occurrences action allows you to find a run of 1 or more spaces and convert it to a tab.

2. TextSoap also supports regular expressions. Beyond the standard regex find and replace, there is also a regex find and apply cleaner which allows you to manipulate the substring via another (custom) cleaner.
Previous Thread | Next Thread
Reply

  Bookmarks  
Digg Diggdel.icio.us del.icio.usStumbleUpon StumbleUponGoogle Google