Thread: Seperating Strings

Hi,
I was trying to learn about dealing with Strings. I came across with StringTokenizer. It seemed like it makes things easier, but I cannot seperate a text if I put a space between words. Here is my code for it:

public class Tokening {
String str;
Scanner input;
StringTokenizer token;
 
       public void methodOne(){
 
		input = new Scanner(System.in);
		System.out.println("Enter some text here:");
		str = input.next();
		token = new StringTokenizer(str,":;.,  "); 
		int tokenCount = token.countTokens();
		System.out.println(tokenCount);
		while(token.hasMoreTokens()){
		String z = token.nextToken();
		System.out.println(z);
		}
 
	}
 
}

When I try this, I put "Here we go, trying to use String,Tokenizer. Hope the best!", it only shows "Here" as the output. That's why I included lines to count the tokens. And it seems I only have 1 tokenizer here.

Looking deeper into seperating strings, I found out that there is a String method called "split". But this didn't help too. I was trying to use more than one delimiters. It failed. I was convinced one would enough, but it produced the same output as StringTokenizer. Here is the code for this - In the same class, so the variables are the same:

public void metotIki(){
		input = new Scanner(System.in);
		System.out.println("Enter some text here:");
		str = input.next();
		String[] stg = str.split(",");
		for(int x=0;x<stg.length;x++){
			System.out.println(stg[x]);
		}

So, I want to seperate text when the user puts a comma, a period and the other signs like ; : ? and I want to do it whether there are whitespaces between the words. How can I do that?

From the StringTokenizer API:

"StringTokenizer is a legacy class that is retained for compatibility reasons although its use is discouraged in new code. It is recommended that anyone seeking this functionality use the split method of String or the java.util.regex package instead."

Stick with String.split(), and give the Pattern API a read-through for regular expression help.

Pattern (Java Platform SE 6)

Haha, I should have read more carefully. Ignore the previous post please
I found the answer to my first question:

public void metotUc(){
		input = new Scanner(System.in);
		System.out.println("Enter some text here:");
		str = input.next();
		Pattern pat = Pattern.compile(":");
 
		String[] stg1 = pat.split(str);
 
		for(int x=0;x<stg1.length;x++){
			System.out.println(stg1[x]);
		}
		}

But what happens when I want to use both : and ; as delimiters? And I can't still seperate the words if I enter "Yes : we are here" instead of "Yes:we are here."

EDIT: OK, I tried it all, I used " ", " : ", ": " and " :" as parameters to String.split() and Pattern.compile() and it didn't worked. So, if my text includes a space (or maybe other whitespaces too), I cannot split it.

Can't you just use something like "[;: ]"?

I'm not really sure what you mean. Would you mind throwing together an SSCCE that shows what you're talking about, with the output you expect vs what you're getting?

That's not really what I meant. You're making this harder than it should be. Try this out:

    public static void main(String... args){
    	String str = "This will be that: a great, colorful, peaceful world: WTF!";
 
    	String[] pieces = str.split("[:,\\s]+");
 
    	for(String s : pieces){
    		System.out.println(s);
    	}
    }

Ohhhh I see.

Scanner.next() gives you the next token, based on a customizable delimiter pattern.
To do it my way, you'd want Scanner.nextLine() instead.

But you could avoid all the String.split stuff if you just used the delimiter pattern in the Scanner in the first place and let it separate them out for you.

Originally Posted by beer-in-box

Okay, so all happened because I was lazy and didn't use nextLine...

More or less, yeah.

Originally Posted by beer-in-box

So, can you examplify what you told please?

I'm not sure what you mean. What didn't you understand?

Originally Posted by beer-in-box

And,in "[:,\\s]+", why are we using "+" and why do we put two "\"s?

Check out the pattern class to see what they are. Write a small program to play with them to see what they do.

Originally Posted by beer-in-box

This :
Can you give me some clues so that I can try to figure it out. My first attempt failed as always

What I meant was to use the Scanner.next() function, but specify the pattern delimiter like you are with the String.split() method. Check out the description at the top of the Scanner API to see how to do that.