I have a server that has been installed for over two years and runs a nightly backup, one of the steps of which is:
mysqldump -u user --password=password --opt --no-create-info databasename table1 table2 table3 | /usr/local/bin/cstream -b1048576 -OD -o /disk2/filename
This worked reliably for a long time, but recently it is failing on a approximately 50% of the backup runs with:
mysqldump: Error 2013: Lost connection to MySQL server during query when dumping table `table1` at row: 4428
table1 is a myisam table with a fixed row length of 32 bytes and 20m rows. Rows are only ever inserted into this table - no updates or deletes. Inserts only occur at a specific time of day and never when the backup is running. It has 7 integer columns and 1 date column, no keys.
I'm using unix sockets to connect to mysql so net_write_timeout isn't relevant.
cstream (http://www.cons.org/cracauer/cstream.html) just writes the data piped in directly to disk with the O_DIRECT flag so that it doesn't put a copy of the dump file in the linux disk cache flushing other blocks that would be more usefully cached. It is functionally equivilent to writing directly to the file otherwise.
/disk2 is a mount point for a 2nd hard disk that is only used to hold backup files, so won't be busy. Previous steps in the backup script will have accessed it so it won't need to spin up.
The failure is intermittent when run repeatedly with the same table content and usually with a very low row number.
max_allowed_packet = 16M
net_buffer_length = 32K
The server has 4Gb of RAM, and usually has around 2Gb in the disk cache so has capacity to run the dump operation without running out of memory.
I've found other articles on the net about this error message but none of them seem to apply to my situation. They either:
- apply only to innodb
- apply only to tcp/ip connections
- are to do with character set/mysql 4.1 compatibilty and my table has no character datatypes
- are for when --quick (or --opt) isn't used in mysqldump
Does anyone have any suggestions?
mysqldump -u user --password=password --opt --no-create-info databasename table1 table2 table3 | /usr/local/bin/cstream -b1048576 -OD -o /disk2/filename
This worked reliably for a long time, but recently it is failing on a approximately 50% of the backup runs with:
mysqldump: Error 2013: Lost connection to MySQL server during query when dumping table `table1` at row: 4428
table1 is a myisam table with a fixed row length of 32 bytes and 20m rows. Rows are only ever inserted into this table - no updates or deletes. Inserts only occur at a specific time of day and never when the backup is running. It has 7 integer columns and 1 date column, no keys.
I'm using unix sockets to connect to mysql so net_write_timeout isn't relevant.
cstream (http://www.cons.org/cracauer/cstream.html) just writes the data piped in directly to disk with the O_DIRECT flag so that it doesn't put a copy of the dump file in the linux disk cache flushing other blocks that would be more usefully cached. It is functionally equivilent to writing directly to the file otherwise.
/disk2 is a mount point for a 2nd hard disk that is only used to hold backup files, so won't be busy. Previous steps in the backup script will have accessed it so it won't need to spin up.
The failure is intermittent when run repeatedly with the same table content and usually with a very low row number.
max_allowed_packet = 16M
net_buffer_length = 32K
The server has 4Gb of RAM, and usually has around 2Gb in the disk cache so has capacity to run the dump operation without running out of memory.
I've found other articles on the net about this error message but none of them seem to apply to my situation. They either:
- apply only to innodb
- apply only to tcp/ip connections
- are to do with character set/mysql 4.1 compatibilty and my table has no character datatypes
- are for when --quick (or --opt) isn't used in mysqldump
Does anyone have any suggestions?