Perhaps make it possible to set a timeout when creating an open challenge. That way you can have it remove itself in a set amount of time, like 20 minutes. This might be easier (and less costly) to implement than either having the challenge feature check for player activity or coding the site such that a player going inactive checks to see whether open challenges exist.
Sure, that could also work.
EDIT: Would be glad to get some (any) feedback on this from Mojko / developers as it seems I must otherwise stop setting up 5 min games for now. It just leads to more lost games than anything. Making the 5 minute option a little pointless unless I am literally on the phone with whoever I'm playing against.