Loading incomplete headers is taking days

bl4h

I just updated a newsgroup that I hadn't checked in about a week. It has now been loading incomplete headers for three days non-stop and it's only done about 500 of the 3500 it reported three days ago. I just closed NMP thinking it was an issue and NMP closed with an error saying it needed to be restarted.

I'm running MYSQL on a completely separate 320gb drive. I have a Core 2 Duo 6800 with 4gb of RAM and a total of 1.5TB of storage across 4 hard drives. NMP is 2.7.0.6 Beta 2.

Any ideas how to stop this process and how to keep it from occurring again. The week of messages only took about 15 minutes to download three days ago.

This is my.ini. Is there any way to increase these values based on my system? Thanks.

# MySQL Server Instance Configuration File
# ----------------------------------------------------------------------
# Generated by the MySQL Server Instance Configuration Wizard
#

[client]

port=3306

[mysql]

default-character-set=latin1

[mysqld]

port=3306

basedir=X:/MySQL/

datadir=X:/MySQL/Data/

default-character-set=latin1

default-storage-engine=MYISAM

sql-mode="STRICT_TRANS_TABLES,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION"

max_connections=100

query_cache_size=20M

table_cache=256

tmp_table_size=100M

thread_cache_size=8

#*** MyISAM Specific options

myisam_max_sort_file_size=10G

myisam_max_extra_sort_file_size=10G

myisam_sort_buffer_size=64M

key_buffer_size=100M

read_buffer_size=4M
read_rnd_buffer_size=4M

sort_buffer_size=8M

query_cache_limit=2M

bulk_insert_buffer_size=32M

join_buffer_size=32M

delay-key-write=ALL

local-infile

max_allowed_packet=64M

;myisam-recover=FORCE

query_cache_type=1
query_cache_size=10M
query_cache_limit=10M

binlog-ignore-db=newsman

#*** INNODB Specific options ***

skip-innodb
_________________
Core 2 Duo E6800, 8gb of PC6400, 4 320gb WD SATA HD's, MSI P965 Platinum, Plextor DVDR/RW 760A, Plextor Premium, ATI X1950XT 256mb PCI-E16, Antec P180B Case, Antec True Power Trio 650w Power Supply

administrator · Posted: Tue Jan 15, 2008 4:17 pm Post subject:

The first thing to do is to move the downloaded header batches out of the batch folder so NMP won't process anymore headers. Then, restart NMP and make sure everything is OK.

Second, if the process of parsing new headers is slow, this usually indicates that your database may be too large to get decent performance on the MySQL side. Cleanup what's in your database, then do a purge and optimize. After this, you can put the header batches back and restart NMP. If the database is in good shape, the process should be back to normal speed.

Regards

bl4h · Posted: Tue Jan 15, 2008 10:27 pm Post subject:

I'm not sure how to move the headers out of my DB? Is it all 3 files labeled wgheaders?
_________________
Core 2 Duo E6800, 8gb of PC6400, 4 320gb WD SATA HD's, MSI P965 Platinum, Plextor DVDR/RW 760A, Plextor Premium, ATI X1950XT 256mb PCI-E16, Antec P180B Case, Antec True Power Trio 650w Power Supply

administrator · Posted: Tue Jan 15, 2008 11:10 pm Post subject:

No, it is the files *.hdr stored in your header batch folder. Open "Settings" and look at the Default Folders tab to see where this is for your machine.

Regards

bl4h · Posted: Wed Jan 16, 2008 9:31 pm Post subject:

Great. Thanks.

Working on it now.

Did my my.ini look ok for a machine with my specs?
_________________
Core 2 Duo E6800, 8gb of PC6400, 4 320gb WD SATA HD's, MSI P965 Platinum, Plextor DVDR/RW 760A, Plextor Premium, ATI X1950XT 256mb PCI-E16, Antec P180B Case, Antec True Power Trio 650w Power Supply

administrator · Posted: Wed Jan 16, 2008 9:42 pm Post subject:

I didn't see any issues with it. If you want to compare it the one I use, you can download it from the Wiki.

Regards