Handling 15,000 CDN Requests/Sec: The Growth Behind jQuery
June 20, 2015 | Justin Dorfman
MaxCDN is proud to present #MaxScale, our new series that takes an in-depth look at how growing tech companies handle scale at the highest levels of bandwidth, storage, analysis and more. In this installment, we chat with jQuery Executive Director Kris Borchers.
The jQuery Foundation is a 501(c)(6) nonprofit trade association devoted to improving the open web and making it accessible to everyone through the development and support of open source software and collaboration within the development community.
jQuery believes open standards are important for the future of the web and that everyone should be able to access the vast content available on the web whether or not they have any physical, mental, cognitive, or technological disabilities that make the Internet more difficult to use.
Kris Borchers, Executive Director of jQuery, working from his home office near Dallas, TX.
Kris Borchers is the Executive Director of the jQuery Foundation. In his role he works daily to help drive the jQuery Foundation’s mission toward achieving a more open and accessible web. Whether he is working with project developers, meeting with current and future supporting members, or even occasionally writing a bit of code, he is always working to grow the foundation and help it bring the greatest positive impact to the open web.
Kris started out as a web developer and software engineer, but always had his eye on finding a way to make the web better. He eventually found that opportunity with jQuery.
Justin Dorfman: Why was the jQuery foundation formed?
Kris Borchers: The jQuery Foundation was formed to be a place for jQuery and its related projects to live and be supported outside of the control of a single person or organization. We have extended that philosophy to start welcoming other projects looking for similar support and neutrality in their governance.
Individuals and enterprises alike prefer when a project they depend on lives in a neutral and stable place. Here they can rely on it being around when they need it and know that its destiny won’t be decided by the whims of an individual or the business plan of some other organization.
JD: What was the original infrastructure behind code.jquery.com? How did it work?
KB: Prior to 2007, people basically linked directly to files hosted at http://jquery.com/src, or used the Google API servers. John Resig saw this trend and established ‘code.jquery.com’ sometime in early April of 2007, which used S3 to store the files.
When Amazon released CloudFront in November of 2008, we immediately flipped over to this service and began seeing improved response time, with a true global reach. As far as I can tell, this was our first “real” CDN experience. Even still, CloudFront didn’t support some features (such as GZIP), so we were still recommending that folks use Google’s CDN.
We disabled hotlinking to jquery.com and other sites entirely back in 2011, except for code.jquery.com. This helped reduce our web server traffic considerably, and helped reinforce the use of our CDN.
In January 2014, we moved to our current location, MaxCDN. At the time, we were pushing about 20Tb/day of downloads, and now we’re at nearly 40Tb/day. That’s a ton of data and we thank our friends at MaxCDN for providing a truly trouble-free experience. The only time we log into the web interface is to check our download stats!
JD: In 2014, jQuery’s core library was being used by more than 61 percent of the top 100,000 sites. Can you explain the components being used today in jQuery’s infrastructure to support such growth?
KB: If you look at the stats on BuiltWith.com, as of today, jQuery core is actually on about 63% of the top 1,000,000 sites in the world and 17% of the entire internet. That is amazing to me every time I see it. The components that keep that running are actually pretty simple (says the guy not maintaining it).
Obviously any code we serve from the CDN is on MaxCDN, but is also duplicated across a couple of other CDNs for redundancy and historical reasons. We have all of our projects’ code on GitHub, and almost all of our sites run on a custom WordPress install across servers that are currently split between DigitalOcean and MediaTemple.
JD: What role does the CDN play in jQuery? And what would be the consequences of removing the CDN from the infrastructure?
KB: Our CDN is a huge part of the jQuery ecosystem. We usually average around 15,000 hits per second to our CDN with 99.8% of those being cache hits. This provides much better performance for those using our CDN as their visitors can use a cached copy of jQuery and not have to download it, thus decreasing load time.
Would the use of jQuery continue without a CDN? Sure. But the CDN provides that extra bit of oomph that folks want when optimizing their web apps. And if the CDN was to suddenly go away, you would see large portions of the web just stop working as sites using jQuery and the CDN would cease to function properly.
JD: The jQuery team is a remote one. How do team members stay connected, organized, and on the same page?
KB: All of our project teams manage themselves in a way that works best for them. Most of the projects meet on IRC. Many of them use the #jquery-meeting channel and the Esprima project uses #esprima-meeting, both of which are on Freenode. We also have a multitude of other IRC channels and all of them are logged and viewable on https://irc.jquery.org.
We have a mailing list that goes out to all project team members to facilitate cross-project and larger, asynchronous conversations. At the foundation level, we also have mailing lists for internal and public discussions and use things like Skype and Google Hangouts for voice/video calls.
JD: Are there any other tools, services, or applications the foundation relies on to help it scale?
KB: I would say one of the most important tools we use is GitHub. All of our projects are on GitHub and we use their Issues system to track foundation level issues and initiatives.
Also, with our high profile and large web presence, we are prime targets for DDoS attacks and had some issues with downtime on our sites last year. CloudFlare came in and helped us out by providing their services to mitigate those attacks and keep us available to the million or so visits we get each day to our sites.
JD: What are three pieces of advice you’d give people trying to start a developer-centric organization with the intention of making it as successful as jQuery?
KB: The theme across all three items is “open.” First, I would say invest in open source. Open source is not the future, it is now. By investing in open source, your return is multiplied by the culture and external commitment to the projects you are interested in without having to go out and find that investment yourself.
Second, trust in open governance. Open source is about more than just being able to view and modify the source of a project. It’s about how that project is run and by whom it is run. Open foundations like the jQuery Foundation provide a neutral place for project communities, including those individuals and organizations that may be competitors in other settings, to come together and drive their projects in a direction that is best for all involved. Not just the individual who owns the project or the organization that may make decisions to improve their bottom line.
Whether your organization is choosing an open project to use or creating an open source project for others, consider choosing projects under this type of open governance or maybe give your project to the community through an open foundation.
Third, and maybe the most important, is to believe in open communities. By building up a solid contributor base along with your user base, the organization will be able to remain relevant with changing market needs and shift quickly through the help of others you may not have even known had an interest in your project.
By becoming a part of the community, contributing to the community, and supporting the community, your end result could be more successful than you could have ever been on your own.
JD: How does jQuery make money?
KB: As a 501(c)(6) nonprofit organization, the jQuery Foundation doesn’t “make money.” That’s not to say that we don’t have revenue and expenses and aren’t in need of investment though.
We are funded by member organizations that contribute money, developers, resources, and time to support our projects and initiatives. We then take those funds and other resources and invest them in things like developer events, infrastructure maintenance, standards participation, project development, and many other initiatives with the common goal of fulfilling our mission to make the web accessible to everyone.
JD: What are your plans for the future? Any new libraries?
KB: Our immediate plans revolve around supporting the current projects everyone knows and loves while finding additional projects and initiatives that can benefit the JavaScript and web communities, help us fulfill our mission, and that need the support of an open foundation or would be better served in a neutral foundation like ours.
We are doing a lot around internationalization through our project Globalize and our involvement in the ECMA 402 standard and coordination with the Node.js community. We are also doing a lot of work in the JavaScript tooling space through Esprima and are involved in the community effort around developing a better AST. There are a number of other projects in the works, or are just crazy ideas right now, but will hopefully be announced soon.
Photography by Genese Gigi Tremel/ The Lady & Rock. All images released under Creative Commons CC by SA 4 license. For publishers and companies, please attribute photos to “Photographer: The Lady & Rock / Source: MaxCDN” and link to this post. For personal use no attribution is needed.<small
Justin Dorfman
Justin Dorfman is the director of developer relations at MaxCDN. He is also responsible for evangelizing the company’s technologies and championing the needs of developers who use the network.
Leave a Reply