Skip to content

Cron Jobs on Google App Engine

As a side project, I’ve been working on a mobile site to track my local bus system. I’ll have a more detailed post about that once it is a fully polished site. Essentially, the site parses some xml with longitude/latitude data and uses a maps api to display the location. All fairly simple, and it’s built on Google App Engine, which means it will be free/cheap to maintain.

I always wanted the app to be able to continually check the xml feeds, which are updated every minute, so I could store the data persistently and then use memcache to handle large spikes in traffic (hopefully my site will have this good problem). In order to do this, I needed to set up a cron job. Well I was in luck as Google recently added support for cron jobs, a feature not previously built in to App Engine. 

Continue Reading »

Posted 14 April 2009 † admin § Uncategorized ‡ Comments (0) ° Tagged: , , , ,

Social search is Twitter’s next killer app? It’s already Aardvark’s killer app

A fun game has been sweeping the Internets. No, it doesn’t involve answering 25 questions on Facebook. It is speculation about Twitter’s future as a profitable business. Advertising Age recently published an article with (well-sourced) speculation of their own:

Certainly there’s an AdWords-like business there, but, as [Todd] Chaffee [an Institutional Venture Partners general partner and a new Twitter backer] told us, Twitter has another “wild card.”

In the future, searches won’t only query what’s being said at the moment, but will go out to the Twitter audience in the form of a question, like a faster and less-filtered Yahoo Answers or Wiki Answers. Users would be able to tap the collective knowledge of the 6 million or so members of the Twitterverse.

“You put a question out to the global mind, and it comes back,” Mr. Chaffee explained. “Millions of people are contributing to the knowledge base. The engine is alive. You get feedback in real time from people, not just documents.”

This is essentially what Aardvark does. And a really good job of it, I might add. Around the same time that Twitter’s biggest goal was to stop crashing, the Aardvark team received $6 million in investment to refine social search. Right now, Aardvark routes its questions and answers between several channels: Google Chat, AIM, Windows Live Messenger, as well as good old e-mail. I posed this question to “assistant curator of birds” of Aardvark, Rob Spiro:

adelevie: @robsp can Vark work through Twitter the same way it works through GChat and e-mail?

12:55 PM Mar 4th from web

His reply:

robsp: @adelevie yup, we’re definitely planning an aardvark-twitter integration, using Twitter as another communication channel… coming soon…

4:24 PM Mar 4th from web in reply to adelevie

Would their two services compete? Could Twitter acquire Aardvark? We’ll just have to wait and see.

Aardvark has recently been ramping up activity, changing the name of their company from Mechanical Zoo to Aardvark, preparing to offer unlimited friend invites to current users, and launching a blog.

Posted 13 March 2009 † admin § Uncategorized ‡ Comments (0) ° Tagged: , , , , , , , , , ,

Better project template for Google App Engine

I’ve been playing around with Google App Engine quite a bit lately. Whenever I’d create a new project, I’d start by copying and pasting a folder called “new_project_template” which is included in the software developers kit. The template contains a bare-minimum hello world script. As I made more and more apps, I realized that I always added the same things to this folder before writing the actual “guts” of the code.

I always had to fix the indents, add some basic import statements, and include my favorite web scraping library, Beautiful Soup. So I created a template of my own and named it “better_project_template” (creative? I know). The folder has the Beautiful Soup module, base.html and index.html templates, as well as some useful shortcuts added to the main.py file. Anyone interested can download this folder here for their own use.

Posted 13 March 2009 † admin § Uncategorized ‡ Comments (0) ° Tagged: , , , ,

No one will know you’re using bit.ly

I’ve been a big fan of Bit.ly. The added utility it provides to link shortening is unmatched. It makes tinyurl.com look like the url shortener of simpletons. Using Bit.ly conveys that you are tech-savvy to a certain degree. But what if you want the benefits of Bit.ly, without looking pretentious? Use biturlly. It’s a simple tool I developed last week that takes a long url, shortens it through Bit.ly, then shortens the Bit.ly link through tinyurl. You’ll get the click tracking of Bit.ly with the humility of tinyurl.

Posted 04 March 2009 † admin § Uncategorized ‡ Comments (0) ° Tagged: , , , ,

Bit.ly: Almost the Delicious of the Semantic Web

A little while back I wrote about how Bit.ly has the potential to bring us ever closer to the “semantic web” comparing it to the seemingly out-of-date delicious.com. Since that post, Bit.ly has added a few features as well as refined its user interface which suggest where Bit.ly may be headed.

The most significant new feature is profile pages. Just as Delicious accounts are in the form of delicious.com/username, Bit.ly accounts are in the form of username.bit.ly. Instead of linking to a Delicious account to show the world what you are sharing, you can just link to your Bit.ly account. However some features are missing from the profile pages, most notably, share/click counts. Bit.ly could easily list next to each link on adelevie.bit.ly the number of times someone has shortened the same url as well as total number of times people have clicked on that link. The data is all there, it’s only a matter of incorporating it into the user interface. Nevertheless, the people at Bit.ly do seem to understand the power of using their own click data to find what is relevant on the web. They’re testing the waters very slowly with this concept.
screenshot comparison of delicious and bitly profile pages

Back in October of 2008, the Bit.ly blog had a post titled Yesterday’s Top Bits where they simply listed the highest-clicked Bit.ly links. Later, in February of 2009, Bit.ly created a Twitter account called bitlynow which tweets the highest-clicked Bit.ly links for a certain time period. This gives Bit.ly the ability to keep a pulse as to what’s being shared on the web in a similar manner as Twitturly. Twitturly’s advantage is that its data include just about every link that goes through Twitter. Bit.ly’s advantage is that its click/referral tracking feature allows it to look beyond just Twitter. I’d like to see some amalgamation of Twitter, Twitturly and Bit.ly.

While Bit.ly’s new features are encouraging, they still don’t address the 800 pound gorilla in the room: its growing database of semantic data about each Bit.ly link (from Open Calais‘ api). Solely tracking the flow of links only gives part of the picture. Tracking the flow of the meaning of those links is a lot more valuable. It would be nice to see Bit.ly make sense of all this data.

Posted 28 February 2009 † admin § Uncategorized ‡ Comments (0) ° Tagged: , , , , , , , ,

Dumb App Engine Shortcuts

I like using App Engine’s webapp framework. It’s very simple and similar to the web.py framework. However, simple tasks such as outputting data or rendering templates require way too much boilerplate text. For example, to output a rendered template, you must type:

1
2
3
4
5
6
7
8

    template_values = {
      'greetings': greetings,
      'url': url,
      'url_linktext': url_linktext,
      }
 
    path = os.path.join(os.path.dirname(__file__), 'index.html')
    self.response.out.write(template.render(path, template_values))

To avoid this morass of boilerplate code, and to better keep track of your code, simply add the following snippet to the beginning of every instance of the RequestHandler class (each “page,” if you will):

1
2
3
4

disp = self.response.out.write
def render(template_name, template_values):
          path = os.path.join(os.path.dirname(__file__), '%s' % template_name)
          disp(template.render(path, template_values))

Now to render a template, simply use return render('index.html', {'form' : someForm()}).
Think of this as App Engine’s version of Django’s Render_to_Response function ;).
Similarly, if all you want to do is spit out data that won’t go through a template just use return disp("some string").

Posted 11 February 2009 † admin § Uncategorized ‡ Comments (1) ° Tagged: , , , , , , , ,