Performance issue. Will DictionaryBase help me?

  • Thread starter Thread starter Gustaf
  • Start date Start date
G

Gustaf

I got an ArrayList with a large amount of objects in (probably no more
than 20000). Every time I add a new object to the array, I need to check
that it hasn't been added already, by comparing two properties from the
new object with every object in the array. It's a slow but necessary
process, because the input files are often overlapping. I'm looking for
ways to make it faster. Someone recommended DictionaryBase. Does anyone
here have positive experiences with it in cases like mine?

Gustaf
 
Gustaf said:
I got an ArrayList with a large amount of objects in (probably no more than
20000). Every time I add a new object to the array, I need to check that it
hasn't been added already, by comparing two properties from the new object
with every object in the array. It's a slow but necessary process, because
the input files are often overlapping. I'm looking for ways to make it
faster. Someone recommended DictionaryBase. Does anyone here have positive
experiences with it in cases like mine?
Click to expand...

You need to use a hashtable and create a key from the 2 properties that
uniquely identifiesthe object. Check if the hashtable contains the key and
add the object if it doesn't. Dictionary implements a hashtable internally
and you get type safety as it uses generics. What were you currently doing?

SP
 
SP said:
You need to use a hashtable and create a key from the 2 properties that
uniquely identifiesthe object. Check if the hashtable contains the key and
add the object if it doesn't. Dictionary implements a hashtable internally
and you get type safety as it uses generics. What were you currently doing?
Click to expand...

It's a small app to consolidate MSN chat logs. The objects I need to
keep in memory are Message objects, and there's a MessageCollection
class already, but it doesn't implement any interface for collections.
Message objects are added to MessageCollection one file at a time with
AddChatFile(). That's where I run into this problem of checking which
messages are already added.

public void AddChatFile(string file)
{
ChatFileReader chatFileReader = new ChatFileReader();
chatFileReader.Load(file);
Message[] messagesInFile = chatFileReader.Messages;
foreach (Message message in messagesInFile)
{
if (!IsAlreadyAdded(message))
this.messages.Add(message);
}
}

private bool IsAlreadyAdded(Message message)
{
if (this.messages == null) return false;
foreach (Message m in this.messages)
{
if (m.DateTime == message.DateTime &&
m.From == message.From)
{
return true;
}
}
return false;
}

Gustaf
 
Gustaf said:
SP said:
You need to use a hashtable and create a key from the 2 properties that
uniquely identifiesthe object. Check if the hashtable contains the key
and add the object if it doesn't. Dictionary implements a hashtable
internally and you get type safety as it uses generics. What were you
currently doing?
Click to expand...

It's a small app to consolidate MSN chat logs. The objects I need to keep
in memory are Message objects, and there's a MessageCollection class
already, but it doesn't implement any interface for collections. Message
objects are added to MessageCollection one file at a time with
AddChatFile(). That's where I run into this problem of checking which
messages are already added.

public void AddChatFile(string file)
{
ChatFileReader chatFileReader = new ChatFileReader();
chatFileReader.Load(file);
Message[] messagesInFile = chatFileReader.Messages;
foreach (Message message in messagesInFile)
{
if (!IsAlreadyAdded(message))
this.messages.Add(message);
}
}

private bool IsAlreadyAdded(Message message)
{
Click to expand...

Here is where you will check a hashtable / Dictionary for a key based on the
DateTime and From properties, e.g. m.DateTime.ToString("yyyyMMddhhmmssffff")
+ m.From should give you a unique key.

SP
 
SP said:
Here is where you will check a hashtable / Dictionary for a key based on the
DateTime and From properties, e.g. m.DateTime.ToString("yyyyMMddhhmmssffff")
+ m.From should give you a unique key.
Click to expand...

Thanks SP. That's a great improvment in performance, and smaller code
aswell. IsAlreadyAdded() could be replaced with a simple if clause.

public void AddChatFile(string file)
{
ChatFileReader chatFileReader = new ChatFileReader();
chatFileReader.Load(file);
Message[] messagesInFile = chatFileReader.Messages;
foreach (Message message in messagesInFile)
{
string key = message.DateTime + "_" + message.Alias;
if (!this.Dictionary.Contains(key))
{
this.Dictionary.Add(key, message);
}
}
}

Gustaf
 
Back
Top