How to upload an older queue_log file
2009-02-20Imagine you have a queue_log file that uploads to partition P003, and is
missing from timestamp 100 to 150; and imagine that old data is
available in a file called queue_log.old
- Make sure there is no data in the table for partition P003 between
timestamps 100 and 150.
e.g SELECT * FROM queue_log WHERE partition='P003' AND time_id BETWEEN 100 and 150 - Stop importing on partition P003
- Move queue_log data newer than timestamp 150 to a new partition (eg P003B)
e.g. UPDATE queue_log SET partition = 'P003B' WHERE partition='P003' AND time_id > 150 - Run the qloaderd on your queue_log.old file, so that it will load all data.
You will have to watch its log file to se when it ends adding lines. - Check that all data has been loaded
e.g. SELECT count(*) FROM queue_log WHERE partition='P003' AND time_id BETWEEN 100 and 150 - Stop the qloader process
- Move back newer data from P003B to P003
e.g UPDATE queue_log SET partition='P003' WHERE partition='P003B' - Point qloaderd to your normal queue_log file, and restart normal importing