Drupal High

Why I love going to DrupalCon

krs — Mon, 07 Mar 2011 11:57:22 PST

Here's the story of me saving at least several hours of development work in a 5 minute chat...

On the flight out to Chicago on Sunday, I was working on a webform addon module for a client (integrating with Exact Target if anyone cares). I got most of the way through it, but I definitely felt like I was doing some things "the hard way".

The next day, during the pre-conference training session lunch break - I grabbed a plate and plopped down at an open seat. Said "hi" to the guy sitting next to me. Nathan "quicksketch" Haug. Who maintains the webform module. Who is super nice and helpful, as many Drupallers are. He indulged my questions and gave me some tips on how to better architect my module.

Hopefully this kind of story helps you convince your boss to send you to the next one! The return-on-investment is very real. These kinds of interactions are incredibly valuable, and happen all the time at conferences, summits and camps.

using RewriteMap

krs — Fri, 30 Jul 2010 12:56:53 PDT

This is an Apache Directive that I've never had to use before, but it came in very handy for a very specific problem.

There was already an apache redirect (RewriteRule + RewriteCond) in place, but the destination URL was case sensitive! That's not normally a problem, but it was for an ad server, and the variables were coming in as uppercase, but needed to be lowercase after the redirect. Bad programming on the part of the ad server in my opinion, but we're not going to let that stop us! :)

RewriteMap to the rescue!

First off, the actual directive is a lot like a function definition, and it can only go in a config file or vhost, it's not allowed in a .htaccess file. Luckily the one we want to use is built in, so we just make it available with:

RewriteMap      lc int:tolower

This makes the "lc" function is available in our rewrite rules. We start off with the condition and basic rule ...

  # Send ads to new ad server

  RewriteCond %{HTTP_HOST} ^old\.ad\.server\.com$ [NC]

  RewriteRule ^.*site=([^/]+)/.*area=([^/]+)/.*$ http://newad.sever.net/ad/$1/$2 [R,L,NC]

To use the RewriteMap function in your RewriteRule just change the $N replacement to ${lc:$N}, and you're all set. So for example $1 becomes ${lc:$1}. In our example above, the new RewriteRule looks like this:

  RewriteRule ^.*site=([^/]+)/.*area=([^/]+)/.*$ http://newad.server.net/ad/${lc:$1}/${lc:$2} [R,L,NC]

Sequential updates with SQL

krs — Fri, 13 Nov 2009 14:14:58 PST

Oh no! A series of posts were uploaded to the server in reverse order!

The site uses a view that sorts posts into chronological order (node created time). We can't change the view just to handle these errant posts. We could update all the node created times one at a time in the database, but there's just too many - it'll take all day!

I knew it would be possible to fix this with a little SQL, but I've never had to do something like this before. After some reading and research, this is what I came up with

SET @rownum := 0;
UPDATE node SET created = UNIX_TIMESTAMP(DATE_ADD(NOW(), INTERVAL (SELECT @rownum := @rownum + 1) MINUTE)) WHERE nid > 100 AND nid < 200 ORDER BY created DESC;

There's a lot going on in this statement:

SET @rownum := 0; initializes the rownum variable to 0.

You can mostly ignore the date stuff - Drupal uses integer timestamps while MySQL prefers real Dates. My plan is to use the DATE_ADD() function to add ever larger amounts to the starting time (NOW()). This could be a lot simpler just leaving everything as integers, but I like the challenge :)

INTERVAL (SELECT @rownum := @rownum + 1) MINUTE

This is really the key part of the statement. We're doing a subselect right in the middle of the UPDATE statement. This causes rownum to be incremented by 1, and the new value returned, which is then used to update just this row. When the next row is updated, rownum will be incremented again, and so on. So when each node gets its created time updated, the first will be now(), the second now() + 1 minute, the third now() + 2 minutes, etc.

WHERE nid > 100 AND nid < 200

The where clause just specifies which nodes we're updating, nothing special but it is critical. Make sure you're only affecting the rows you want to update.

ORDER BY created DESC

This is the other key part. Ordering for updates works just like it does for select statements. All the rows are getting updated, this lets us specify in what order the updated happen.

So now when we put it all together, the magic happens. The nodes were uploaded in reverse order, remember? That means when we sort by created time in descending order, we get the newest ones first - those are the ones that should be the oldest. So when the update happens, the newest node gets the oldest time, and second newest gets the second oldest, ... all the way until the oldest node gets the newest time.

And so they end up with reversed created times, and the view will now show them in the correct order.

Drush beats CVS, part 2

krs — Wed, 05 Aug 2009 12:14:27 PDT

So it seems like a lot of people disagreed with my previous post! :) I should mention that I'm not against CVS and the title may have been provocative, the cvs_deploy module works very well, and CVS and SVN are not mutually exclusive - you can have both in the same repository and they will happily ignore each other.

But it got me to thinking, what am I missing? I use SVN repositories for the sites that I build. I use Drush to install the modules that I need and check for upgrades. The module upgrade process is slower for me since I like to know all the details of what changed first. So I'm open to people letting me know how I could be working smarter, and how using the CVS respository could be making my life easier!

I know the comments system at onSugar is not the greatest, feel free to email me via my Drupal contact form, and I'll post the responses.

Drush beats CVS

krs — Thu, 30 Jul 2009 17:14:10 PDT

I had checked out a bunch of modules fron CVS, but then I realized that CVS checkouts don't come with all the info such as version numbers, so I wanted to delete them and replace them with real copies. I used Drush and Bash, and here's how I did it.

Need to delete every CVS file from the sites/all/modules directory, and then replace each module with a fresh copy downloaded from drupal.org, and do it without ruining your SVN repository? "What's the difference?" you ask. Well, modules checked out directly from CVS don't get extra packaging info from drupal.org, and don't keep track of their own version numbers properly (you'll see a blank Version column on admin/build/modules).

So by getting a fresh copy from drupal.org, you get a proper .info file, and a LICENSE.txt file added for you. And it's a long module list? No problemo!

Assumptions - you have Drush installed and you're working with a checkout of an SVN repository.

cd to your modules directory, usuallycd examplesite/sites/all/modules

Update your modules to their latest versions - Drush is going to try and download the latest anyway, and you want them to match up.

Tell Drush to grab fresh copies of modules from drupal.org with:
find . -name CVS -depth 2 | sed 's/\// /g' | awk {'print $2'} | xargs drush dl
and watch the "Project xxx downloaded to ..." messages scroll by.

Get rid of all the CVS files with find . -name CVS -prune -exec svn rm {} \;

Add all the new files (should just be a bunch of LICENSE.txt files) with
svn status | grep '^?' | xargs svn add
Then do a quick svn status - and you should see a bunch of Deletes on CVS files, Adding LICENSE.txt files, and Modifying any .info files

Last, SVN commit the whole bunch! yay!
svn commit -m "removed old CVS tags, redownloaded modules from drupal.org" .

edit: thanks to Joel for a better introduction

update_sql is not my friend

krs — Wed, 07 Jan 2009 22:51:53 PST

Oh update_sql(), second most unloved of the drupal database functions (db_rewrite_sql() is worse)! If you've never used it, it is basically a wrapper for db_query() that you can use in hook_update_N() hooks in a module .install file. You may never need to write one of these hooks unless you maintain a module; or like me you prefer to make database changes via code when I'm pushing some new features out to dev, stage, and live servers.

The Drupal 6 and 7 versions of the documentation for this function do mention that "%-substitution parameters are not supported." Definitely also true for the Drupal 5 version I'm still working with. So you can't do variable substitution like you do with db_query() and most of the time thats not a big deal.

There is one very important time though.

When using db_query(), you call it with a syntax like db_query($sql, $variable1, $variable2); Now, you know about database prefixes, and always sing curly braces around your table names in FROMs and JOINs, and that lets Drupal work on multisite installs very well, or talk to more than one DB at a time. So the $sql part of that function call always gets passed through a strtr(), or in English, PHP's string translate function that converts every instace of '{' to the right prefix (generally just deletes it). You have no choice about this, and no way to prevent it from getting every single { and }!

So far so good, because you only use curly braces around the table names, and that works out just fine, right?

Now enter update_sql(). This function doesn't support variable substitution, so your function calls look are just update_sql($sql).

Like update_sql("UPDATE {node} SET title = 'something else' WHERE nid = 10"); - that is, everything has to be passed as part of a single SQL statement (the $sql part above). The SQL statement that always gets run through strtr() looking for '{' and '}'. Which, if you are trying to update a serialized array, or a block's visibility code with PHP, or just want to use a { in a node title, it will be deleted. Every time, without exception.

So if you're using update_sql() and any part of the data contains a '{' or '}' that you need to keep intact - rewrite it as a db_query() with variable substitution. The strtr() will operate on the $sql part, and the $values get substituted in later and stay intact.

link fields and tokens

krs — Wed, 07 Jan 2009 22:30:13 PST

Did you know the CCK Link field can accept tokens in the link title? I didn't. Now the link module provides all kinds of ways to format the combination of url + title that get entered, but when asked to have the actual node title link to the given url - I thought for sure I would have to resort to writing some code, either in template.php or in a tpl file.

But the Link module and Token module together made it easy. Set the Title to "static title" - this way the user will never be prompted when creating a node, and remember this is the link title, not the node title. Then set it to the token [title-raw] and you're done! CCK and possibly Views will all work together to get you something like <a href="[user entered url]">[node title]</a>!

How to hook_views_tables_alter()

krs — Wed, 07 Jan 2009 00:50:21 PST

First of all, this only applies to Views1 & Drupal 5 - there's probably a similar function in views2 however.

The setup: A timestamp ($node->created) was being selected in a View. The only options for formatting it are as Short, Long, Medium, Time Ago, or Custom. With custom you can enter any string that can be interpreted by php's date() function, and have that format apply.

The problem: The site that used a different format as its standard for displaying date and time, let's say "D F j, Y \a\t g:i A". We had created a function in template.php called site_format_date() that accepted a timestamp, and returned the date nicely formatted that way. All for reuseability! Now all the dates in the code (and template files, and theme function overrides) we write can call that function. And when Mrs. Editor decides that she really would like it to be something completely different, we can change it in one place and be done!

The table problem: What a pain to have to go to each View that uses a time or date, and select the "custom" format, and enter into the textbox the format we want to use. Now when someone wants to change the site-wide format, we'll have to go back to each view and update all the format fields! Argh!

So what we're trying to do, is somehow add our own custom format as one of the options for displaying a timestamp, in addition to the basic ones Drupal and Views provides us. Unfortunately, those choices are created by a call to views_handler_field_dates(), which just returns a fixed array of those 5 choices, and there's no way to get your own in there.

hook_views_tables_alter() to the rescue! This hook lets you modify how the Views module understands the tables in the Drupal database. Most of the core tables are added by .inc files in the views module, and many contrib modules have their own views.inc file. Using this hook you have the opportunity to change anything you like about how views works with modules, and do it safely without hacking any modules.

The fields I wanted to affect (the created and changed timestamp) are node fields. You can see them defined in path/to/views/modules/views_node.inc. I'm trying affect how the fields are handled (as opposed to filters, arguments, sorts, or anything else). By checking the .inc file I can see the keys I want to affect are simply called 'created' and 'changed'. Putting that all together, here is my complete hook function :

function hook_views_tables_alter(&$tabledata) {
$tabledata['node']['fields']['created']['handler']['my_custom_format_date'] = t('As My Custom Date');
$tabledata['node']['fields']['changed']['handler']['my_custom_format_date'] = t('As My Custom Date');
}

Now when I can choose "As My Custom Date" to format the timestamps! Views will automatically call my_custom_format_date() for me, and I just pass the data right along to our sitewide date formatter.

function my_custom_format_date($fieldinfo, $fielddata, $value, $data) {
return $value ? site_format_date($value) : theme('views_nodate');
}

If figuring out the syntax of the array to modify was a little confusing (views syntax? confusing? never!) try this little snippet first:

function hook_views_tables_alter(&$tabledata) {
drupal_set_message(print_r($tabledata, TRUE));
}

and you'll get a very long printout of how every table interacts with the Views module. With some searching and little luck, you should be able to find the section you want to modify. Lots of great views1 and views2 documentation is available, but it gets a little hairy! Unfortunately there's not a lot of documentation that I could find on this very powerful hook! I discovered it while poking through the views.module code and found 1 Drupal issue at http://drupal.org/node/142383. (This was the issue that actually enabled the hook to work properly as of about July 2007).

Remind me to add some pictures to this post!

Imagecache and You (err, your picture)

krs — Tue, 06 Jan 2009 23:21:42 PST

When you using imagecache on the $user->picture, many people have noticed 1 small problem. If someone uploads a new picture, everything appears to work fine, but the user photo stays the same!

What's going on? When a user uploads a picture, it gets renamed to a specific filename - doesn't matter the name of the file you uploaded. Imagecache then makes a copy of that file to display. (Technically this doesn't happen until the first time the image is viewed with a particular preset). When the new user image is uploaded, it gets the very same name as the previous one, again regardless of the actual filename that was uploaded. Now imagecache will think it already has a cached version of that image, because it just checks the filename and sees that it exists!

So I was all about to post on how to fix that, basically by linking to Nate Haug's excellent post on how to do just that - but the Imagecache module maintainers have just made a new release (for Drupal 5) that includes a fix for the issue! So the new solution is - download the latest version of the Imagecache module!

Ampersands and CCK

krs — Tue, 06 Jan 2009 22:58:15 PST

When using a CCK text field that is set to display as a select list, using the "Allowed Values" textarea to specify each of the choices works 99% of the time. But when trying to use an option that included an ampersand, like "Drupal & Drupal", it keeps getting escaped to & - stop it CCK!

What's going on? Each line of the textarea is eventually passed through filter_xss(), which tries it's best to remove naughty characters like &, becauseof its special meaning in HTML.

How to avoid it? Use the section "PHP Code" and write your values out as an array. You must specify a key and value pair for each option, CCK will not automatically handle just having the value. Don't forget to return the array. PHP open and closing tags arent needed. CCK trusts the values found in this block, and pass them through any kind of text filtering, and your ampsersands will sail through undamaged.

Drupal & Drupal
Hans & Franz
Martin & Lewis

become

return array(
'Drupal & Drupal' => 'Drupal & Drupal',
'Hans & Franz' => 'Hans & Franz',
'Martin & Lewis' => 'Martin & Lewis',
);