Sequential updates with SQL

krs — Fri, 13 Nov 2009 14:14:58 -0800

Oh no! A series of posts were uploaded to the server in reverse order!

The site uses a view that sorts posts into chronological order (node created time). We can't change the view just to handle these errant posts. We could update all the node created times one at a time in the database, but there's just too many - it'll take all day!

I knew it would be possible to fix this with a little SQL, but I've never had to do something like this before. After some reading and research, this is what I came up with

SET @rownum := 0;
UPDATE node SET created = UNIX_TIMESTAMP(DATE_ADD(NOW(), INTERVAL (SELECT @rownum := @rownum + 1) MINUTE)) WHERE nid > 100 AND nid < 200 ORDER BY created DESC;

There's a lot going on in this statement:

SET @rownum := 0; initializes the rownum variable to 0.

You can mostly ignore the date stuff - Drupal uses integer timestamps while MySQL prefers real Dates. My plan is to use the DATE_ADD() function to add ever larger amounts to the starting time (NOW()). This could be a lot simpler just leaving everything as integers, but I like the challenge :)

INTERVAL (SELECT @rownum := @rownum + 1) MINUTE

This is really the key part of the statement. We're doing a subselect right in the middle of the UPDATE statement. This causes rownum to be incremented by 1, and the new value returned, which is then used to update just this row. When the next row is updated, rownum will be incremented again, and so on. So when each node gets its created time updated, the first will be now(), the second now() + 1 minute, the third now() + 2 minutes, etc.

WHERE nid > 100 AND nid < 200

The where clause just specifies which nodes we're updating, nothing special but it is critical. Make sure you're only affecting the rows you want to update.

ORDER BY created DESC

This is the other key part. Ordering for updates works just like it does for select statements. All the rows are getting updated, this lets us specify in what order the updated happen.

So now when we put it all together, the magic happens. The nodes were uploaded in reverse order, remember? That means when we sort by created time in descending order, we get the newest ones first - those are the ones that should be the oldest. So when the update happens, the newest node gets the oldest time, and second newest gets the second oldest, ... all the way until the oldest node gets the newest time.

And so they end up with reversed created times, and the view will now show them in the correct order.

update_sql is not my friend

krs — Wed, 07 Jan 2009 22:51:53 -0800

Oh update_sql(), second most unloved of the drupal database functions (db_rewrite_sql() is worse)! If you've never used it, it is basically a wrapper for db_query() that you can use in hook_update_N() hooks in a module .install file. You may never need to write one of these hooks unless you maintain a module; or like me you prefer to make database changes via code when I'm pushing some new features out to dev, stage, and live servers.

The Drupal 6 and 7 versions of the documentation for this function do mention that "%-substitution parameters are not supported." Definitely also true for the Drupal 5 version I'm still working with. So you can't do variable substitution like you do with db_query() and most of the time thats not a big deal.

There is one very important time though.

When using db_query(), you call it with a syntax like db_query($sql, $variable1, $variable2); Now, you know about database prefixes, and always sing curly braces around your table names in FROMs and JOINs, and that lets Drupal work on multisite installs very well, or talk to more than one DB at a time. So the $sql part of that function call always gets passed through a strtr(), or in English, PHP's string translate function that converts every instace of '{' to the right prefix (generally just deletes it). You have no choice about this, and no way to prevent it from getting every single { and }!

So far so good, because you only use curly braces around the table names, and that works out just fine, right?

Now enter update_sql(). This function doesn't support variable substitution, so your function calls look are just update_sql($sql).

Like update_sql("UPDATE {node} SET title = 'something else' WHERE nid = 10"); - that is, everything has to be passed as part of a single SQL statement (the $sql part above). The SQL statement that always gets run through strtr() looking for '{' and '}'. Which, if you are trying to update a serialized array, or a block's visibility code with PHP, or just want to use a { in a node title, it will be deleted. Every time, without exception.

So if you're using update_sql() and any part of the data contains a '{' or '}' that you need to keep intact - rewrite it as a db_query() with variable substitution. The strtr() will operate on the $sql part, and the $values get substituted in later and stay intact.

Drupal High

Sequential updates with SQL

update_sql is not my friend