Twitter logged me as someone else! Privacy FAIL!

October 28th, 2010

A while ago, clicking on the “tweet” button of a funny FAIL in Google Reader gave me the scare of my life.

First reaction: How did my twitter theme change?
Second reaction: When did i start tweeting/reading in Thai?
Third reaction: When did I change into a pretty girl?

I was logged in as someone else! Thats probably what brain transplant(when possible) would feel like…

In other words, the transparent proxy at True simply fucked up… can’t do anything about it. There is probably someone else in Thailand having the time of his/her life browsing into my account and poking into my disgusting life…

So, the solution for me would be to use a secure tunnel bypassing True’s evil session hijacking transparent proxy all together.. Which is kinda illegal I’ve been told… and also who will protect me from all the porn(and evil propaganda) on the interwebs ;)

@kiqq_3112 : Ive tried to censor sensitive stuff from the screenshots, if you are offended by anything, give me a shout out ill remove it. Just wanted to show how serious is this issue.

Click on the images to see full size full page… (Private tweets were censored)

1) Woah I’m not me I’m her?!?!?!

Twitter security FAIL

2) I can tweet as her!! (Note: I didn’t actually click the tweet button)

Twitter security FAIL

3) Spy on DMs!

Twitter security FAIL

4) Replies Page..

Twitter security FAIL

5) Change settings..

Twitter security FAIL

6) Change password!!!!

Twitter security FAIL

Download all images
Disclaimer: I haven’t broken into anyone else’s account, I don’t do such things. My photoshop skills are not good enough to be able to fake these. Moreover I don’t even own a copy of photoshop.

Whos.amung.us goes down taking out numerous client websites

September 13th, 2010

Realtime web analytics service whos.amung.us has been completely down for at least the last 3 hours (as of sept 12 4:28pm UTC). Thousands of webmasters (myself included) use this widget to gather realtime traffic stats for websites. The current outage has caused big problems to the sites implementing this widget.

Lets look at their widget implementation.


<script type="text/javascript" src="http://widgets.amung.us/small.js"></script><script type="text/javascript">WAU_small('unique-site-tag')</script>

This is bad!

The base javascript uses document.write, hence cant be made to load asynchronously. Moreover, they write an img tag into the document which too must be be loaded before the window.onload event can be triggered.

This incident was somehow not critical FAIL for me cause I am aware about frontend SPOF issues and placed this widget just before </body>, so it didn’t completely mess with my page, but still caused the following problems.

  1. Delayed the window.onload event, delaying executing of other scripts made to run after onload.
  2. Showed “waiting for …. ” in the status bar of the browsers for a long time before timing out, indicating to the user that the page is still not ready.
  3. Makes Google think your site is really slow and doesn’t deserve to rank!

Now, for webmasters not aware about these issues the problems is possibly critical. In case the widget code is before the main content of their site, they are in for a hard time. Their site would stop loading at the point the widget is located in the html, giving the users an impression that the website is broken.

I understand that today is a Sunday, and not all the engineers/OPs can be on standby 100% of the time to fix these issues. Downtime is inevitable, but when you FAIL, don’t take your customers/users down with you. Their javascript serving server was still more stable than the server. Here is a Webpagetest.org result for my site during the outage. Its the dynamic counter image which SPOF’d.

If they must use document.write, write a div, give it a name, and attach the image into it using dom manipulation after the window.onload.

Moreover, when the servers come under high load, there should be a mechanism to FAIL early with an appropriate status code.

Respect your users, don’t take them for granted!

Disclaimer: My current secret project is somewhat related to real-time analytics.

Optimizing DFP performance

September 11th, 2010

Ive been using DFP (DoubleClick for Publishers – formerly Google Ad Manager) to serve ads as it has some certain features I really need.

The downside of using DFP is that its browser side performance sucks big time. On my site, I use the experimental Iframe tagging, which basically writes an iframe in place of the ad and loads the third party ad codes into it. This runs asynchronously and is nice, but the DFP’s bootstrap javascript blocks rendering and it needs to load 3 javascript files from Google serially. This bootstrap must be available before trying to fill an ad slot.

The Goal

Show the story title and body to the user As Soon As Possible

The Problem

The instructions of implementing the Iframe tagging is as such.

In the <head>…</head> section :-


<!-- PUT THIS TAG IN THE head SECTION -->
<script src="http://partner.googleadservices.com/gampad/google_service.js" type="text/javascript"></script>
<script type="text/javascript">
GS_googleAddAdSenseService("ca-pub-xxxxxxxxxxxxxxxx");
GS_googleEnableAllServices();
</script>
<script type="text/javascript">
GA_googleUseIframeRendering();
</script>
<!-- END OF TAG FOR head SECTION -->

And where the ad slot needs to be displayed always in the <body>…</body>:-


<script type="text/javascript">
GA_googleFillSlotWithSize("ca-pub-xxxxxxxxxxxxxxxx", "slot_name", 728, 90);
</script>

The resulting waterfall chart of this method :-

DFP before

Requests #3, #4 and #5 is the bootstrap for DFP. The green line indicates the first paint event.

It is clear that

  1. The browser needs to make 3 sequential requests
  2. Until these 3 javascripts are downloaded and parsed, the rendering cannot begin. – The user is affectively staring at a blank screen.
  3. The browser is doing nothing else while downloading these files.

The Workaround

I was going to mark this section as a “solution” to the problem… but it isint. The page is still susceptible to frontend SPOF .. That is if the servers hosting the DFP JavaScripts is unaccessible, that calls for a terrible user experience on my site.

What I did was simply moved the bootstrap to just before the first time the GA_googleFillSlotWithSize() function was called. i.e. move the bootstrap from the <head> to the <body> part of the page.

The waterfall chart for this is :-

DFP after

Requests #3, #7 and #8 is the bootstrap for DFP. The green line indicates the first paint event.

It is clear that

  1. The browser still needs to make 3 sequential requests
  2. While the bootstrap was loading, the user can see the header of the site, providing visual feedback that something is happening. – The start render did not have to wait.
  3. The browser is downloading images, etc referred to earlier while downloading the bootstrap.
  4. There is 200ms improvement in the time at which the user can start reading the story.

Here is a video comparing the 2 loading methods :-

If anyone has better ways to embed DFP, please let me know via comments below.

The Solutions

There are 2 ways to solve this

  • Have source ordered HTML where the adcodes are always after the content in the HTML source of the page. This ensures that the user can start reading while DFP does its blocking thing. – A site redesign is in planning and that would implement this step.
  • DFP changes the way it works… by replacing the evil document.write with DOM manipulation techniques so the scripts can be loaded asynchronously.

So next time if a third party script provider tells you that certain code must be in the <head> of the HTML, don’t believe them and question them.

PS: All tests were done from ie7 from Dulles, VA using WebPageTest.org, ie8 also showed similar behavior.

Shameless Plug

I am available for consulting on web speed issues, contact details in the right sidebar.

UPDATE: I am now using Aaron’s adsense hack to defer DFP below the content. Thanks Aaron
UPDATE 2: New Post: Complete Asynchronous ad loading using DFP and LABjs

Twitter finally gets something right – Tweet Button

August 13th, 2010

FAILing all the time is not new for Twitter, in fact they are the masters of FAIL. Twitter FAILs so often that seeing the obnoxious FAIL WHALE on a daily basis has become a way of life for most of us. Twitter is so experienced in the art of FAIL, that they are aware and experienced in causes of FAIL and how to mitigate its effects.

Recently Twitter, for the first time ever, launched the official Tweet button which one can put on their websites to allow the user to tweet easily and view the twitter based popularity of the page they are currently reading.

So whats new about this? 3rd providers like TweetMeme have had this functionality for years now. Let me explain …

Firstly Twitters implementation of the Button


<a href="http://twitter.com/share" class="twitter-share-button" data-count="vertical">Tweet</a>
<script type="text/javascript" src="http://platform.twitter.com/widgets.js"></script>

Look closely, there is an <a> tag which sets the class attribute, and then a <script> tag, which makes the magic happen. Take a look into their JavaScript, there are no nasty document.writes in it. In fact, all it does is look for objects with the twitter-share-button class and then that object accordingly. Now, if the <script> is placed just after <a> as advised by Twitter, the page load blocks while the JavaScript is downloaded and parsed, this in certain cases may drive away impatient users who don’t like waiting for stupid buttons to load before they can interact with rest of the page.

Since Twitter’s JavaScript doesn’t use document.write, the <script> can be included anywhere in the html provided it is after the <a> tag. the ideal position would be to place it just before </body>. The result, adding this 3rd party widget induces no additional block in the rendering of the page. Implementing the Tweet Button in this manner has negligible impact on pageload.

This is how your page would look.


<html>
<head>
[...]
</head>
<body>
[...]
<a href="http://twitter.com/share" class="twitter-share-button" data-count="vertical">Tweet</a>
[...]
<script type="text/javascript" src="http://platform.twitter.com/widgets.js"></script>
</body>
</html>

To make it asynchronous, the <script> tag can be implemented as such :-


<script id="deferedjs" type="text/javascript">
var b = document.createElement('script');
b.type = 'text/javascript';
b.src = ('http://platform.twitter.com/widgets.js');
var a=document.getElementById("deferedjs");
a.parentNode.insertBefore(b,a);
</script>

Now thats is truly non-blocking un-intrusive implementation of the button. It doesn’t block the onload event uselessly.

Now, Twitter didn’t include these methods in the documentation probably because they didn’t want to scare away regular bloggers and webmasters who aren’t as paranoid about client side performance as some of us.

The Tweet Button uses Akamai CDN to serve the assets, which is considered super stable. Currently sets an expires header of 1 hour (which I presume will be changed to far-future once things stabilize) and to top it off, it provides webmasters with an elegant JavaScript which can be implemented such that the base page suffers very little(if any) even if their CDN goes down.

More experienced webmasters can even bundle this JavaScript into their own code, provided they regularly track changes and re-bundle often to avoid undesired consequences.

Now, see TweetMeme’s implementation :-


<script type="text/javascript" src="http://tweetmeme.com/i/scripts/button.js"></script>

Just take a look into their JavaScript. It uses a document.write to write an Iframe. The JavaScript has to be downloaded, then parsed, then executed exactly at the moment it needs to be placed in sync with the rendering of the DOM. There is no way that I know of to make this truly asynchronous besides putting their script inside another iframe (which makes one additional request to your server).

TweetMeme hosts the javascript on their own servers, if their server is slow, the whole pageload would suffer. To top it off there is no way where a webmaster can take precautions against it.

My intention was not to single out TweetMeme, almost all 3rd party widgets make their code such that each of them are frontend SPOF for the websites that use them.

Moral of the story is that it is the responsibility of a 3rd party service providers to make their FAIL shouldn’t make their users sites to FAIL completely.

Page Speed Analysis: Search Engine Roundtable

July 31st, 2010

I am planing to do a series of posts on analyzing popular websites I visit frequently. By now I am so obsessed to optimize for speed that I’ve run out of sites to optimize, hence I’m doing it for other sites.

To start with I’ll analyze Search Engine Roundtable.

Before I start, wanna make it absolutely clear that I am not looking to make the site owners or webmasters look bad. Barry Schwartz, editor of SERoundtable.com, tells me “The site is old and is being completely revamped from bottom up.”

About SERoundtable.com :-

Traffic Rank: 5,738 (Alexa), 16,804 (compete)
Browsers : Assuming lesser than average Internet Explorer users based on the nature of the target demographics.

My assumption is that the site targets regular (loyal) readers, so I place more emphasis on repeat views and caching.

The Problem

The website is slow, I access it regularly and loose productivity while waiting for the pages to load.

Page Load times (using WebPageTest.orghere and here):

Page Start Render Load Time
Home Page – First View 1.894s 9.733s
Home Page – Repeat View 0.914s 4.619s
Individual Post – First View 1.539s 11.855s
Individual Post – Repeat View 0.858s 5.171s

Web Speed Rankings (Using Show Slow):-

Page Yslow grade Page Speed score
Homepage D (65) C (74)
Individual Post D (64) C (75)

Visual Latency (tested from Dulles, VA using IE7):

Homepage : 2.5 seconds
Post Page : 2.6 seconds

(The timing I’ve used here is the time at which the page becomes somewhat usable, this in my definition is the time at which the content title and body is shown on the screen. This is the time at which the user can start engaging with the site while other stuff loads.)

Thus you can see that for whole 2.5 seconds the user is doing nothing but staring at an almost blank screen.

So we have a problem here, unoptimized websites open exponentially slower from across the globe in Thailand especially when using crappy ISPs. So whats the solution?

The Solution

Firstly, kudos to SERoundtable.com for using Source Ordered Content(SOC). This ensures that the title and text of the post is the first thing to render on the page, lesser susceptible to Frontend SPOF. The reason for SOC here may or may not be for SEO purposes, but in terms of page speed this is a valuable optimizations. Also try to move the adslot SER-HOME-468×60-1 to below the content.

Now the things that needs to be changed (In order of importance as I see it) :-

  1. Minimize the number of requests : The time taken to load the page is somewhat directly proportional to the number of requests. Most people these days use high speed internet access which basically gives them a higher bandwidth but not better speed (latency). The following steps need to be implemented to reduce the number of requests.
    1. Disable Etags and add far futures cache headers for static objects (Easy/Urgent/Obvious): 49 of the 50 requests made to www.seroundtable.com are non frequently changing static files (CSS, images, etc). Currently the browsers send a If-Modified-Since requests to all of them to check if it has changed since then or not. My advise is to send expires headers to at least 7 days( > 30 days preferred) in the future. This would make repeat views amazingly fast. Quite easy to implement cache control in Apache. Etags don’t serve much purpose here, require a tiny bit of CPU for each request, useless information IMHO.
    2. Use CSS Sprites (Easy – Intermediate): Request for following files can easily be sprited (after converting everything to either gif or png – png prefered) — nav_previous.gif, nav_next.gif, h2.gif, 3dots.gif, pen.png, sphinn-favicon.png, Twitter-16×16.png, digg-favicon.jpg, barry.png, entry-footer.png, comforminput.gif, comformtextarea.gif, sbauthors.gif, sbarchives.gif, sbmoreinfo.gif, mbg.jpg, sbhome.gif, sbourforums.gif, delicious-favicon.png, google-favicon-16.png, sbtheforums.gif, arrow_yellow.gif, rss.gif, foo.jpg, credits.jpg – Thats a reduction of 24 requests!
    3. Use Data-URI for the tiny background files (Advanced/Optional): Embed the tiny background files directly inside the CSS using Data URI Scheme. The drawback is IE7 and below dont support this, the workaround is to use the star hack to send them the current background settings. Another drawback is this would increase the size of the CSS file, but the total size after compression(see below) would be about the same as the current total size. This optimization reduces the need for the following files in all non IE browsers and IE8 — sbbg.gif, formbg.gif, mbg.jpg, sbhbg.gif, sboxbg.gif, tbbg.gif, rcredbg.gif, rcbg.gif, threadlibg.gif, bg.gif, hbgleft.jpg, msep.gif, fbg.gif – Thats elimination of 13 requests! Another advantage is that the css file starts to download much before actual rendering starts, this at the start of rendering, these files would already be available, making the background graphics appear almost instantly without additional blocking of rest of the page.

    So Out of 50 requests made to www.seroundtable.com, 37 of them can be completely eliminated for first view, for repeat views almost all but base page should be cached.

  2. Enable Keep-Alive(Easy/Obvious): An uncached pageload makes about 50 requests to www.seroundtable.com. For each of these requests, the client must make a new connection with the server. Enabling Keep-Alive can get the job done in much fewer(depends on browsers parallel connection policies) connections, thus saving dozens of round-trips. Enabling keep-alive does increase memory utilization on the server, but the benefits are greater. See this chart, the brown portion of the loadtime can be removed for ~90% of the requests. Here are some instructions for Apache.
  3. Optimize each request : Reduce the size of each request/response using the following methods
    1. Gzip text assets (Easy/Obvious): About 75% of data transfer for the base page and the css file can be reduced by simple enabling gzip on Apache.
    2. Compress images(Easy) : My favorite tool for this is Smush.it by Yahoo! Similar results can be achieved using say GIMP or Photoshop.
    3. Use cookie-less domain for static content(Easy/Important): Serve static files(images/css/etc) from an external domain. Currently, the site sets cookies which the browser automatically sends with each subsequent request to the server while requesting static files. This cookie information is absolutely useless to the server and causes the request size to be big. The workaround is to host these files on another dedicated cookieless domain, say www.serassets.com or something. This tip may not sound important, in fact I used to ignore it, but once implemented the effects are HUGE.
    4. Minify code (Easy/Optional): Minify HTML, run the inline javascript using Google’s Closure Compiler for faster execution. Smaller files also require lesser resources on the browser.
    5. Use a CDN(Easy): A CDN (Content Delivery Network) mantains POPs (Point Of Presence) globally, and can be used to serve the static files(images, javascripts, css, etc) which would be served from a server closer to you. The charges are not much. So if SERoundtable.com used say Internap, I would be accessing the static files from a server in Singapore instead of US. In case of Akamai I’d be accessing the static assets from a server within my ISPs network! Choose any from this list of CDNs.
  4. Onpage/rendering optimizations:
    1. Choose <link> over @import (Easy): Yahoo! explains it better than I can. Affects usability in IE, delays rendering.
    2. Move the inline JavaScript into an external file (Easy): This would make the page load faster for repeat users since the base page would be lighter and the JavaScript file would already be cached in the client’s browser (or at upstream proxy). Make sure to minify the JavaScript (see point 3-4)
    3. Move JavaScript to the end (Intermediate): The JavaScript files/codes that are not necessary to be accessed during the page rendering, should be moved to the bottom of the page just before or just before they are needed.
    4. Use the Async Google Analytics Code(Easy): The new method really doesn’t block the rest of the page. The current document.write method in the section blocks rendering on many browsers.
    5. Move DFP code lower(Intermediate): Right now, the javascript code for DFP is placed within the section, this makes 3 serial blocking requests which must be downloaded and executed before rest of the elements download and render blocking the page for 500 to 1000 ms. My suggestion is to move this just above the first Adblock(the GA_googleFillSlot() calls) in the HTML. This combined with the current Source Ordered Content architecture will ensure that the users will feel minimal impact even if DFP has problems.
    6. Try using DFP’s Iframe tagging (Easy but Experimental/Risky): The major drawback to this is that it may mess up expandable ads, which I don’t think SERoundtable.com uses. If you notice the video above, notice starting from about the 4 second mark, currently the 6 adblocks load one by one. With the Iframe tagging, they would load independently, not blocking each other or the rest of the page. A major protection from frontend SPOF due to 3rd party ad networks. Personally I am impressed with the results of this method.
    7. Domain Sharding (Advanced): Split requests across several domains like static1.seroundtable.com, static2.seroundtable.com, static3.seroundtable.com, etc to increase parallel downloads. The downside is that this results in increased DNS lookups which is bad. Fortunately there is a workaround to this issue.

Conclusions

Apart from the tips above make sure to read the best practices for faster websites. Most of what I’ve suggested is redundant from the best practices, tailored to this particular website. Lots more can be done, but at the moment compare to what I’ve mentioned above, the effects of additional optimizations may seem like a waste of time.

Just a reminder that I’ve been told that SERoundtable.com is revamping the entire site and it “is really a full revamp from the CMS to the html on the frontend.”.

Implementation of most of the above tips would make at least one reader of SERoundtable.com very happy :) .

Total time spent : >= 5 hours.

Appreciate any criticism of my analysis via comments below.

A word of caution to other optimizers, don’t get over obsessed with yslow, Google page speed or page load times, its the visitors experience which matters the most. Optimize for visual latency more than bragging rights!

Shameless Plug

I am available for consulting on web speed issues, contact details in the right sidebar.