Search  
Register   Login Sunday, February 05, 2012
 
ForumForumMy Photo Index ...My Photo Index ...Bug reportsBug reportssingle import cache or mdb file limitsingle import cache or mdb file limit
Previous Previous
 
Next Next
New Post
 9/4/2009 11:37 AM
 

Hi,

Trying out version 1.24.3454.

"Stress testing" failed. I was trying to import 46137 files (~11 G total) . Cached image size setting -- 640x480. All files are JPGs of various size.

First try: importing via "Tools->Import->from disk". Only 12248 files imported. No errors, just "these files were not imported" message. MDB file size  ~11 Mb, cache file ~575 Mb

Second try: importing via "drag & drop". Only 12925 files imported. No errors. MDB file size  ~11 Mb, cache file ~575 Mb

Both times I could add new files later.

If this could be fixed -- this is just half of my collection, will be able to test "more stressfull" situation :)

New Post
 9/5/2009 1:21 AM
 

Hi,

Thanks for the stress testing, I am missing this kind of feedback from users.

There is no limit on the mdb file(at least not at this scale) and the cache limit on the version you have is 100,000 images(there is no good explanation why I chose 100,000, just a bad decision) but the next version (1.24 next beta) will set the limit to 1,000,000).

I've tried importing last night (you reely challenged me  so I had to duplicate my collection) and I've successfully imported 33747 images (no failures). (this is maybe due to fixes done in this area but I'm not sure.

Are you importing form a local disk or a network drive?

I will put the new Beta (A lot of changes including a new Personal web interface) in a couple of days, and maybe then you can give it another shot.

Thank you & Regards,

Asaf.

 

 

New Post
 9/5/2009 8:45 AM
 

Hi,

I'm importing from external USB HDD.

Waiting for new beta :)

Guoda 

New Post
 9/12/2009 9:06 AM
 

Hm. Strange -- beta 1.24.3541, import via drag & drop. Using exactly the same file structure. Again 12925 files imported --> think this means it stopped on the same file and couldn't continue.

Log file (BTW, log file time values has nothing in common with system time.. Guess that's why I thought "nothing is there" for the first time):

2009.09.12 12:42:32 - Windows XP Service Pack 3
2009.09.12 12:47:46 - System.Threading.ThreadAbortException: Thread was being aborted.
at System.Threading.Thread.SleepInternal(Int32 millisecondsTimeout)
at MyPhotoIndex.ImageBank.ApplicativeImageCache.CachingThread()
2009.09.12 12:54:59 - System.ArgumentOutOfRangeException: Year, Month, and Day parameters describe an un-representable DateTime.
at System.DateTime.DateToTicks(Int32 year, Int32 month, Int32 day)
at MyPhotoIndex.Utilities.ImageMetaData.ExifExtractor.GetDate(String stringValue)
at MyPhotoIndex.Utilities.ImageMetaData.ExifExtractor.buildDB(PropertyItem[] parr)
at MyPhotoIndex.Utilities.ImageMetaData.ExifExtractor..ctor(String fileName)

<... lots (~80) of "System.ArgumentOutOfRangeException: Year, Month, and Day parameters describe an un-representable DateTime." identical errors. >

2009.09.12 13:05:18 - System.ArgumentOutOfRangeException: Year, Month, and Day parameters describe an un-representable DateTime.
at System.DateTime.DateToTicks(Int32 year, Int32 month, Int32 day)
at MyPhotoIndex.Utilities.ImageMetaData.ExifExtractor.GetDate(String stringValue)
at MyPhotoIndex.Utilities.ImageMetaData.ExifExtractor.buildDB(PropertyItem[] parr)
at MyPhotoIndex.Utilities.ImageMetaData.ExifExtractor..ctor(String fileName)
2009.09.12 13:34:00 - System.NullReferenceException: Object reference not set to an instance of an object.
at MyPhotoIndex.Utilities.ImageMetaData.IPTC.ExtractIPTC(Stream fs)
2009.09.12 13:34:01 - System.NullReferenceException: Object reference not set to an instance of an object.
at MyPhotoIndex.Utilities.ImageMetaData.IPTC.ExtractIPTC(Stream fs)
2009.09.12 13:34:01 - System.NullReferenceException: Object reference not set to an instance of an object.
at MyPhotoIndex.Utilities.ImageMetaData.IPTC.ExtractIPTC(Stream fs)
2009.09.12 13:34:01 - System.NullReferenceException: Object reference not set to an instance of an object.
at MyPhotoIndex.Utilities.ImageMetaData.IPTC.ExtractIPTC(Stream fs)
2009.09.12 13:34:01 - System.NullReferenceException: Object reference not set to an instance of an object.
at MyPhotoIndex.Utilities.ImageMetaData.IPTC.ExtractIPTC(Stream fs)
2009.09.12 13:34:01 - System.NullReferenceException: Object reference not set to an instance of an object.
at MyPhotoIndex.Utilities.ImageMetaData.IPTC.ExtractIPTC(Stream fs)
2009.09.12 13:35:02 - System.OverflowException: Value was either too large or too small for a UInt32.
at System.Convert.ToUInt32(Int32 value)
at MyPhotoIndex.Utilities.ImageMetaData.ExifExtractor.convertToInt32U(Byte[] arr)
at MyPhotoIndex.Utilities.ImageMetaData.ExifExtractor.buildDB(PropertyItem[] parr)
at MyPhotoIndex.Utilities.ImageMetaData.ExifExtractor..ctor(String fileName)

<about 33 "System.OverflowException: Value was either too large or too small for a UInt32" identical errors>

2009.09.12 13:41:44 - System.Threading.ThreadAbortException: Thread was being aborted.
at System.Threading.Thread.SleepInternal(Int32 millisecondsTimeout)
at MyPhotoIndex.ImageBank.ApplicativeImageCache.CachingThread()

Found the file where import stopped. Nothing strange there -- scanned as jpg, RGB 8 bit mode; 556 KB.

Tried to append it as single file to those 12925 files -- no errors, file got imported.

Deleted all 12926 files. Tried to import all the directory with that "strange" file -- no errors, everything got imported.

New Post
 9/13/2009 2:39 AM
 
 Modified By Asaf  on 9/13/2009 1:40:51 AM

Hi, this is strange,

I'll go over the error handling in this area again(the file even if erroneous should not stop the whole import process.

Regards,

Asaf.

Previous Previous
 
Next Next
ForumForumMy Photo Index ...My Photo Index ...Bug reportsBug reportssingle import cache or mdb file limitsingle import cache or mdb file limit

 
Membership Membership:
Latest New User Latest: russell97
Past 24 Hours Past 24 Hours: 0
Prev. 24 Hours Prev. 24 Hours: 0
User Count Overall: 443

Online Now Online Now:
 
del.icio.us del.icio.us | technorati technorati | reddit reddit | stumbleupon stumbleupon | facebook facebook | live live
Copyright 2006-2011 by My Photo Index - the free open source photo organizer