Feed aggregator

Draft Redpaper, last updated: Wed, 22 Feb 2012

- Introduction to IPv6
- IPv6 Addressing and Packet Format
- Host Configuration - Windows, Linux, AIX, and VMware

Anyone that follows information technology has heard about the Internet running out of IP addresses.

Draft Redpaper, last updated: Wed, 22 Feb 2012

- Analyzing bottlenecks Critical path analysis Applying the latest z/OS features

Batch performance optimization remains an important topic for many companies today, whether merging workloads, supporting growth, reducing cost or extending the online day.

Below is the presentation I gave at this year’s 2012 Cloud Connect in Santa Clara.  It was extremely well received.  Better than I expected really, given it’s last minute nature.  For some, I think a lot of the architectural and design patterns aren’t new, but perhaps they haven’t been portrayed in quite this way before.  For others, a lot of these ideas and content are new.

Regardless, I got some props from Adrian Cockcroft, one of the primary cloud architects for Netflix.  I think very highly of Netflix approach to re-architecting their application to be ‘cloud-ready’ and to achieve high levels of uptime on AWS EC2.  If Adrian thinks this is right on, then it must be.

Enjoy.

 

Architectures for open and scalable clouds

View more presentations from Randy Bias

Cocomore (via DrupalFire)

I was already planning to provide an overview of what’s been going on in the various Drupal 8 initiatives even before last week, when Dries announced the timeline for Drupal 8, which includes a “feature freeze” for Drupal 8 in only a little more than nine months from now, and planned release at the DrupalCon Europe, in late August 2013.

Drupal 7’s Plateau of Productivity?

While most of the top Drupal 6 modules are now available, in some state or another, for Drupal 7, and I would certainly choose Drupal 7 for a large Drupal-based project that is not expected to be deployed for some time, from the outcry of protests I think there are a lot of people who would not agree that Drupal 7 is yet at its Plateau of Productivity. There is still plenty of reason to build sites on Drupal 6, especially if you need particular features (e.g Nodewords / Metatag functioning properly, among others) and if you need to deploy the site now, with those features ready for use. Dries indicated that he thought Drupal 6 reached its Plateau of Productivity in late 2009, about 18 months after its initial release. At that point, there were fewer than 20,000 sites using Drupal 5 and more than 200,000 sites using Drupal 6. While this order-of-magnitude-greater-usage is not likely to ever be seen comparing Drupal 7 vs Drupal 6 usage (at least not before Drupal 8 is released), I do think that it’s significant that Drupal 7 usage has finally overtaken Drupal 6. That said, I don’t think we are truly at Drupal 7’s Plateau of Productivity, the point where building a new site on Drupal 6 would be “pointless”. Both in terms of time-after-release and usage statistics, it is arguably premature to say we are quite to that point yet.

Quibbling aside, I also don’t think it matters whether we believe we have reached the Plateau of Productivity for Drupal 7, or not — and it certainly doesn’t matter whether we are all in agreement about that. I do think we are ready to see core development for Drupal 8 get kicked into high gear and I don’t think it will significantly delay the development of the contrib modules or resolving core issues in Drupal 7 which are the final barrier, in my view, to truly reaching its Plateau of Productivity. Additionally, many of the fixes and features going into Drupal 8 are regularly being back-ported to Drupal 7, and there is increased discussion of relaxing the criteria for what can be back-ported to Drupal 7, so I see the increased attention to Drupal 8 core development as exciting: a win-win for the whole Drupal community. We now have a release date for Drupal 8, which is important for business decisions, and a better timeline to facilitate a roadmap for the final stages of determining feature inclusion.

Drupal 8 Core Initiatives

Currently there are 6 official Drupal 8 Core Initiatives which are working on various aspects of desired improvements to core. There are others likely to be added to the list as soon as a bit more progress has been made on the current list and/or as qualified individuals step up to take on some of the other “top 10” desired improvements we had on our collective community wishlist. Some of the improvements require fixes to issues plaguing Drupal 7 and 6 and have been backported. Most of the others involve dozens, if not hundreds, of related issues. Following is a brief summary of each of the current core initiatives and what their priority goals are for Drupal 8. In the interest of brevity, the explanations leave out a lot of juicy details, but for those who haven’t been paying close attention and who might like to get involved, I hope this summary is useful:

Web Services and Context Core Initiative

The Web Services and Context Core Initiative (WSCCI, pronounced “Whiskey”), formerly referred to as the “Butler” project, is a core initiative led by Larry Garfield of Palantir.net, aka “Crell” on Drupal.org. While the traditionally typical HTTP request has been for HTML pages, the modern Web has brought with it the need for HTTP services which deliver information which is not necessarily in the form of HTML. This is especially true for mobile applications, but also applies to feeds and other communications via HTTP. The goal is to “take Drupal from being a first-class Web CMS to being a first-class REST server which includes a first-class Web CMS”. Really, this initiative spans a huge range of related issues and without writing an article many times the length of this one, I could not possibly cover everything, but…

read more

Matthew Saunders (via DrupalFire)

Want to save $50? If you're planning on going to Drupalcon Denver today is the last possible day to get in at the early bird pricing. At midnight Mountain Time, the price goes from $350 to $400. That extra $50 in Denver would a REALLY nice meal for a couple of people with drinks. It could get two people into the official party. It is an Association membership with $20 left over. Come on? What are you waiting for? REGISTER today to access more than 104 sessions including core conversations, three GREAT keynotes, more Birds of a Feather conversations than you can shake a stick at. That does not even include the networking opportunities.

If you register today, that comes to $70/day. That increases by $10/day tonight (February 21/2012) at midnight.

REGISTER FOR DRUPALCON DENVER NOW!

read more

Whether you are in New Orleans today in person or just in spirit, have a Happy Mardi Gras!

 

Larry Garfield (via DrupalFire)

As Dries has already reported, we held a summit meeting at the Acquia offices in Boston last week. It was a good sprint for a couple of reasons. For one, a large number of leading core developers got more clearly on the same page about the direction of Drupal core. For another, we were able to break the "too big to swallow" logjam that has been plaguing the Web Services and Context Core Initiative (WSCCI).

read more

Lullabot (via DrupalFire)

Since version 2.0 of the Views module shipped back in 2008, site builders have been able to use its Exposed Filters feature to create slick user-filterable lists without writing a lick of code. Unfortunately, complex views with lots of exposed filters can easily become cluttered -- some filtering options only make sense when others are also selected, for example. Views Dependent Filters solves that problem, allowing to hide and show exposed filters based on other filters' values.

Configuring the module is a bit opaque: using it requires adding a "Global Dependent Filter" to your existing view, then positioning it between the two exposed filters whose behaviors should be linked. For example, you might add an exposed Content Type filter, then the Global Dependent Filter, then an second exposed filter that's only applicable to one of the content types. The Dependent Filter's configuration options will allow you to choose which values from the first filter should hide or show the second exposed filter.

Redpaper, published: Mon, 20 Feb 2012

- Understand how to attach XIV to IBM i through VIOS
- Learn how to exploit the multipathing capability
- Follow best practices for the number of LUNs and their size

This IBM® Redpaper™ publication explains how you can connect the IBM XIV® Storage System server to the IBM i operating system through the Virtual I/O Server (VIOS).

Redpaper, published: Mon, 20 Feb 2012

- Learn about enhanced performance through shared memory
- See how to increase server utilization with memory savings
- Follow scenarios that show setup and configuration

This IBM® Redpaper™ publication introduces IBM PowerVM® Active Memory™ Deduplication on IBM Power Systems™ based on IBM POWER7® processor technology.

Redpaper, published: Mon, 20 Feb 2012

- Learn how to assess IT infrastructure pain points
- See how to assess the business value of the zEnterprise System
- Understand how to make platform placement decisions

Examining a possible deployment or migration of an application or workload to the IBM® zEnterprise™ System requires a structured approach.

Outlook emailing client is used by most of the organizations for sharing and managing various crucial information among all the employees. Sometimes, Outlook users face unwanted problems in Outlook and then, decide to use Outlook to Lotus email conversion ( ...

Last week, I reviewed ABBYY Business Card Reader, a $5 app that uses sophisticated OCR to read the names and numbers off of a business card so that you can create a contact entry without having to type all of that information by hand.  Florida attorney Katie Floyd of the great Mac Power Users podcast posted a comment on that post suggesting that I check out a similar app called CardMunch.  CardMunch also lets you scan business cards with your iPhone, but it takes a very different approach.

First, the app is free; the professional social network LinkedIn owns the app and gives it away.  More about this in a moment.

Second, as reported by Matt Lynley of Business Insider, the app doesn't use optical character recognition (OCR) to read your business cards.  Instead, it uses a service owned by Amazon called Mechanical Turk, a service that hires real people to do simple, repetitive tasks.  The pay is low, but Mechanical Turk workers have very flexible hours and can work at home, so for many it is an easy way to supplement their income.  Because real people look at the image of the business card and then type the information, the accuracy of CardMunch is much higher than the OCR used by ABBYY Business Card Reader.

Here is an extreme example of how valuable this can be.  A while back, Apple co-founder Steve Wozniak (who goes by "Woz") gave me his business card, and it is unlike any other business card you've ever seen.  It is made out of stainless steel and is laser-cut.  I've heard it said that you could cut a steak with this business card, and I suppose that is possible.  When I tried to scan the card with ABBYY Business Card Reader, the app simply couldn't recognize it.  No surprise; it is etched metal and difficult to read.  Even more curious, the phone number is presented in a format similar to a standardized test with numbers punched out on a grid.  However, when I scanned it with CardMunch, the Mechanical Turk worker was able to figure out that it was Steve Wozniak and sent me back an entry, although I see that even that person couldn't figure out the 408-888-8888 phone number from the card.  (Note: I'm not giving out anything private here; Woz has advertised that number ever since he acquired it in the 1970s back when he used it as a dial-a-joke number.)

 

On the other hand, you need to have Internet access for CardMunch to read a card, and the scan process can take much longer.  In my tests it took about 15 minutes, but I've seen others post on the Internet that it sometimes takes several hours.  ABBYY Business Card Reader, by comparison, can read a card in few seconds.  Having said that, the CardMunch app doesn't make you wait.  Just scan one or more cards and let the app send off the images.  You can then return to the app minutes or hours later and you will notified if you have new contact entries that have been processed.

Third, because the app is owned by LinkedIn, the app (or perhaps more accurately, the Mechanical Turk employee) will try to match the business card with a resume on LinkedIn.  If CardMunch finds a LinkedIn entry, your contact will contain information from the person's LinkedIn profile.  For example, you can see in that scan of the Woz business card that it provides me with information not on the card itself, such as the fact that he is a Felow at Apple, his photograph, etc.  It is a little strange — some might say creepy? — to scan a simple business card and get back a contact entry that includes additional information such as a photograph of the person, employment history, schools attended, etc.  On the other hand, this is all public information that the person has decided to put on LinkedIn, so you shouldn't feel too voyeuristic, and it can quite useful to have all of this information regarding the person who just handed you a busines card.

Herr is the result from scanning my own card, which led to results from my own LinkedIn profile plus phone numbers from my card.  I'm not sure why the Mechanical Turl worker who read my own business card called the main number at my office my "home" number; it is clearly labled "Main" on the card itself.

If the person cannot be found on LinkedIn, then you just get back a simple contact entry that only contains the information from the card that the Mechanical Turk worker was able to read.

CardMunch doesn't have some of the advanced features of ABBY Business Card Holder such as the ability to sort by different attibutes and the ability to assign groups to different contacts.  On the other hand, just like the ABBYY app, CardMunch does give you the option of keeping the contact in the app itself and/or sending the contact information to your iPhone's Contacts.

Note that when you share from CardMunch to the main Contacts app you don't get all of the LinkedIn information such as the profile photograph, website, biographical information, etc.; it appears that the app only share the information taken from the card itself.

Katie Floyd warned me about checking out the Settings app on the iPhone and then scrolling down to the entry for CardMunch.  There is an option called "Auto Connect" and if you have it turned on, every time the app scans a business card of someone who has a profile on LinkedIn, it automatically sends a request to become a LinkedIn Connection.  You might not want to become a connection of the guy trying to sell you life insurance who just handed you a business card, so make sure that this is turned off if you don't want to use the feature.  (For me, it was turned off by default.)

I am torn on whether to recommend ABBY Business Card Reader or CardMunch.  I suppose that it is nice that CardMunch is free versus the $5 for the ABBYY app, but when an app is free that just makes me wonder what else they are doing to make money off of me.  What does LinkedIn do with the information on the business cards that are scanned?  I don't know the answer, and frankly I'm trying to decide how much I even care.  It's not like I often receive business cards from private celebrities or political disidents who are trusting me to keep their contact information confidential from a third party liked LinkedIn, but I can't help but wonder about this.

As for the use of Mechanical Turk and the use of LinkedIn profiles, there are clear advantages to getting scan results that are not only more accurate, but in many cases even more complete than the information on the card itself.  On the other hand, if speed is important, you'll want an app like the one from ABBYY that performs the OCR itself in a matter of seconds. 

There are other apps in the App Store that let you scan and handle business cards with the iPhone and I don't plan to do an exhaustive review of all of them.  Nevertheless, the approach taken by CardMunch is so different from the ABBYY apps that I reviewed last week that I thought it would be interesting to compare and contrast them.  Whichever app you choose, you'll save yourself a lot of time and hassle entering contact information manually, and you have the helpful ability to decide whether you want the contact to remain in the app or be moved to your main Contacts database.

Click here for CardMunch (free): 

Click here to get ABBYY Business Card Reader ($4.99): 

Click here to get ABBYY CardHolder ($2.99): 

SQL Data Types Constant Value SQLCHAR 1 SQLNUMERIC 2 SQLDECIMAL 3 SQLINTEGER 4 SQLSMALLINT 5 SQLFLOAT 6 SQLREAL 7 SQLDOUBLE 8 SQLDATE 9 SQLTIME 10 SQLTIMESTAMP 11 SQLVARCHAR 12 SQLBINARY 2 SQLSQLVARBINARY 3 SQLLONGVARCHAR 1 SQLLONGVARBINARY 4 SQLBIGINT 5 SQLTINYINT 6 ...

As I've mentioned in the past, I'm now writing a monthly column for the TechnoLawyer BigLaw newsletter.  In this week's issue, I wrote about key iPhone apps that law firms should encourage their lawyers to use and the ways that law firms can purchase apps for their lawyers, including using Apple's Volume Purchasing Program.  If you don't subscribe to the BigLaw newsletter yet, I encourage you to do so.  Click here to sign up for a free subscription to BigLaw.  And now, the news of the week:

• If you use FileMaker as your database program, check out this article for TidBITS by Steve McCabe about the FileMaker Go app for the iPhone and iPad.
• Portland, Oregon attorney Josh Barrett of TabletLegal reviews OfficeTime, a time and expense app for lawyers.
• I recently reviewed TranscriptPad, an iPad app that is great for reviewing deposition transcripts.  Yesterday, Josh Barrett posted this interesting interview with Ian O'Flaherty, the developer of the app.
• AppleInsider notes that Apple made $1 out of every $5 spent on consumer electronics in the U.S. this past holiday season.
• New York attorneys Katherine Helm and Joel Cohen discuss the risks of taking your electronic devices to other countries in this article for Law.com.
• Attorney Tom Mighell of the iPad4Lawyers site compares OnLive Desktop and CloudOn, two options for running Microsoft Word on the iPad.
• Boston Attorney Martha Sperry reviews Remarks, a PDF annotation app for the iPad, on her Advocate's Studio blog.
• Kelly Hodgkins of TUAW reviews the 2.0 version of Tweetbot.  Because of Tweetbot's great new features such as inline viewing of images and Readabiilty integration, it is now my favorite Twitter client for both the iPhone and iPad.  I had been using Twitterific.
• David Pogue of the New York Times discusses the next version of the Mac operating system, OS X 10.8 Mountain Lion, which incoporates many concepts from the iPhone/iPad into the Mac and adds better compatability between the iPhone/iPad and the Mac such as a Messages client.
• Alan of Art of the iPhone analyzes whether AppleCare+ for the iPhone is worth it.
• Ron Johnson had been the head of Apple's successful retail stores.  A few months ago, he left Apple to become the CEO of J.C. Penny and he is working to turn that company around.  Johnson recently appeared on CBS This Morning.  The interview begins with him talking about hiring Ellen DeGeneres as a new spokesperson, a decision in the news because it was opposed by a few homophobic protestors, but he also discusses his time at Apple and what he learned from Steve Jobs that he is bringing to J.C. Penny.  The video is embedded in this MacNN article, as is a funny clip from DeGeneres.
• iDownloadBlog has the winners of the 2011 Original iPhone Film Festival.
• And finally, at the recent Grammy Awards, Eddy Cue, Apple's senior vice president of Internet Software and Services, accepted a Grammy on behalf of Steve Jobs for his contribution to music.  Here is Cue's short acceptance speech:

This weeks’ re-launch of Cloudscaling was amazing. It was all we could have expected and more. My only regret was not being able to walk the halls at Cloud Connect as much as I would have liked, but I think I made up for that during our reception, which was a blowout. A weeks’ worth of hallway conversations in only 3-hours time.

That being said, no launch goes flawlessly. We had some misfires early on and some controversy right after the new website launch. I have a well deserved reputation for being controversial in the cloudosphere so it would probably be a very disappointing launch if I didn’t cause at least a *little* stir.

I thought it made sense to address some of these questions and concerns, while also owning up to any of the criticisms I felt were warranted.

Open + Cloud
We took a little heat from Chris Hoff, a friend a fellow ‘clouderati’, about so-called ‘openwashing.’ I think some of Chris’ criticisms are fair[1], but mostly I think his concerns highlight a failure to communicate on our part. This is the great thing about public forums and market feedback. It makes you a better company and frankly those that are sincere and authentic can thrive in that environment. I believe that’s Cloudscaling and so I would like to explain briefly the disconnect.

Chris’ argument boils down to a perceived overloading of ‘open’ as meaning ‘scalable’, such that my litmus tests for ‘open cloud’ are bogus. Except, we aren’t claiming that open is scalable. We’re claiming that ‘open cloud’ is scalable. It’s not open applied to cloud. It’s open AND cloud, where by cloud we mean scalable infrastructure a la AWS.

As long time readers know, I’ve always been uncomfortable with the term ‘enterprise cloud’. Two years ago I was crystal clear about my thinking and haven’t changed my position since. The basic gist is this: so-called ‘enterprise clouds’ aren’t really clouds, they are virtualization 2.0 cloudwashed til it glows. At Cloud Connect 2011 (last year), my keynote reiterated that enterprise clouds are a myth.

Hey, give me points for consistency!

Here’s the problem: everyone with an ‘enterprise cloud’ believes it’s a cloud and their vendors tell them it’s a cloud[2]. It’s hard to throw this term away as much as I would like to, but in this case I need to use the term everyone else does in order to get my point across. In this way it reminds me of the co-opting of ‘hacker’ in the general populace. All of the old skool UNIX folks were unhappy with it, trying to push ‘cracker’ instead, but generally failed. ‘Hacker’ is now synonymous with black hat[3] types. Enterprise clouds will ultimately be synonymous with virtualization 2.0, not clouds in the long run.

So, here’s where we failed. We didn’t make it clear that we are talking about open (freely available, no-vendor-lock-in, preferably open-licensed) PLUS cloud (elastic, scalable, secure, robust). This is our fault. The challenge being, of course, it’s difficult to have any kind of nuanced discussion while creating bite-sized marketing that is easily consumed and tastes great.

No problem. We’ll work on making that part more clear, but hopefully you can see that there is no disconnect between what we’ve said all along, how we are positioning, our marketing message, or are choice to push for a greater understanding of what open clouds are.

Open or Open First as a Strategy
For some, our announcement is seen as a way to justify their early positioning. That’s fine. We think that many people can build open clouds and open cloud products. It’s an umbrella we welcome others to be a part of, even non-OpenStack vendors and solutions companies.

For others, it’s easy to see Cloudscaling ‘jumping on the open bandwagon’. It’s really not for me to say if we are ‘openwashing’ and I’m not going to claim to have been ‘open first’ (does it really matter??). The marketplace will determine that and folks will get their pound of flesh as they see fit. From where I am standing we have been consistent and true.

We are committed to open-ness, open source and open projects. Both my credentials and Cloudscaling’s predate our announcement by a long time. Here’s a taste:

• I open-licensed the GoGrid API in January 2009
• We were part of the original OpenStack launch in 2010 and I wrote about it shortly after
• Cloudscaling stood up the first OpenStack Storage (Swift) production public system outside of Rackspace in January 2011 mostly on relatively open hardware and open networking gear [4]
• Our new VPE, hired fall of 2010, ran OpenSolaris development for years; that’s not an accident

More importantly, we see being open as a legitimate business strategy. Perhaps Dave Asprey’s analysis at the TrendMicro cloudsecurity blog sums it up best. We are pushing open cloud because we think it matters AND we’re open because we think it’s the right business move.

Understanding Production Open Clouds
Cloudscaling’s engineering and architecture teams includes myself, key back-end engineers from GoGrid and other large scale systems (e.g. eBay). We also lean heavily on Chris Brown (the original lead developer for Amazon’s EC2) and Ben Black (one of the core AMZN/AWS network architects) who are both on our technical advisory board. Cloudscalers know a few things about building production-grade cloud services.

All of the above is public information. The following is internal Cloudscaling lore that you’ll have to take my word on.

In early 2009 my engineering team lobbied hard to build our own IaaS software stack, just like OpenStack. They had the experience and DNA to do this. It would be go number 3 or 4 for some of them. The problem was that there was a long list of commercial and open source entities out there already with IaaS stacks. We were a young business and I didn’t want to be IaaS stack #26. The other major problem is that we knew what it takes to build a production cloud service and allocating a VM to a hypervisor was the trivial part. The much harder problems are in operations and ongoing management and still under appreciated area.

So, I resisted and believe me, people were unhappy. Worse yet, I asked them to use one of the existing commercial stacks, now open source, that was poorly architected Java and just about had a revolt.

When I found out about OpenStack in mid-2010, I was relieved. This was what we needed. An open source project with real momentum creating a greater community we could be a part of. I didn’t want to own the creation of the IaaS stack, I just wanted to participate and make someone else’s stack better. That was when the first parts of our generally open strategy started to form. That was also when we knew our future course was software + services. It didn’t hurt that Cloudscaling had a bunch of Pythonistas and that OpenStack was built around Python.  We also felt that the core Nova and Swift architectures were sound.

Summing It Up
We don’t claim to be the most open company out there. Openness is a spectrum and we want to be as open as possible. Other than what we are doing already (see below), we will continue to learn and evolve. We plan to monetize our business outside of the core open components of OCS. We are going to give back network architecture blueprints and documentation, hardware blueprints, open source software, and a raft of other ‘trade secrets’ we have kept close for a long time. I want everyone to build awesome open and scalable clouds. I expect that Cloudscaling will be one of the best and we’re fine competing with the other great teams out there.

The truth however is that we have been interested and committed to open for a long time because our approach is a full-stack approach and we can’t build it all. The only way to succeed is to leverage what exists, which is other open projects. What we plan to monetize is outside of the core open components so there is no conflict. We aren’t open to be cool. We’re open because we care, we want to make great products, and we want to compete effectively against folks in the marketplace. Just like everyone else.

So, feel free to criticize, and we welcome the feedback, but I prefer you do it in context.  Disagree with my definition of cloud if you like.  Meanwhile, we’ll get to work fixing some of our marketing content.

No harm, no foul.

[1] They also deserve to be addressed en masse; however, for this article, I’m choosing to stay focused on the core issue: “open”
[2] Not only this, but honestly, I’ll probably just as criticised for walking around slinging the ‘truth’ in our marketing material that enterprise clouds aren’t ‘true cloud’; damned if you do and damned if you don’t.
[3] Sorry, I still can’t bring myself to use the term ‘hacker’ since it’s been co-opted and even the Wikipedia page is a travesty in that it’s taken on the popular term. C’est la vie.
[4] Amongst a number of others notable production OpenStack deployments, both storage and compute, some of which I can’t talk about unfortunately, but the Intertubes will mostly tell you if you search for it.

Metal Toad Media (via DrupalFire)

In the world of sorting, sometimes 'newest first' or 'oldest first' just doesn't cut it. During a recent Drupal project, we had a client who wanted to be able to control the order of their marquee images in random ways via a drag and drop interface.

Enter the Draggable Views module. In about thirty minutes, I was able to set up custom drag and drop functionality for several content types on their site. Let's dive in and I'll show you how it's done. If you prefer a video tutorial, skip to the end of this post.

• Read more

Content in this document was produced in collaboration with IBM Collaboration Solutions; and IBM® Red books®. IBM® Collaboration Solutions and IBM Redbooks® have partnered together to show you how to create an integrated Web Experience with IBM Lotus Domino, IBM Sametime, IBM Connections, IBM ...

Dries Buytaert (via DrupalFire)

Last weekend, we held a sprint at the Acquia offices for the Web Services and Context Core (WSCCI) Initiative for Drupal 8. This was an important sprint for the future of Drupal. This blog post provides a high-level overview of what was discussed and agreed upon; the different sprint participants will be laying out more technical details in follow-up blog posts.

Overall, a wide range of experience levels, skill sets, and perspectives were brought to the table, with the goal of the sprint being to clearly define the initiative’s scope, get agreement on what we wanted to accomplish and why, and lay out a clear plan for how to accomplish this.

In attendance were:

• Larry "Crell" Garfield, lead architect of the WSCCI initiative
• Daniel "sun" Kudwien, top Drupal core contributor
• Fabien "fabpot" Potencier, lead developer of the Symfony project
• Kris "EclipseGc" Vanderwater and James "neclimdul" Gilliland, who have helped develop and implement the plugin system as part of WSCCI
• Greg "heyrocker" Dunlap, initiative lead of the Configuration Management Initiative for Drupal 8
• Moshe Weitzman, long-time Drupal core contributor
• Angela "webchick" Byron, community cat herder and Drupal 7 core co-maintainer
• Randy "rfay" Fay, Justin "beejeebus" Randall and Alex "effulgentsia" Bronstein, core developers and sane voices of reason with an outside perspective
• Dries Buytaert (me), project lead

Scope

The WSCCI initiative, as envisioned by Larry Garfield, was originally set to address Drupal's web services and flexible page layout capabilities. We discovered that both would require significant changes to Drupal core, and it was difficult to build consensus online, so we decided to get together for 3 days and to flesh out what we actually wanted to accomplish, and how.

At the sprint, we first attempted to articulate all of the problems that WSCCI was trying to solve, which included: multiple page layouts, better UI/UX to manage blocks, partial page rendering (ESI, AHAH), contextual blocks, different response types per delivery callback/URL, plugin system / swappable subsystems, lazy loading bootstrap (convert subsystems to PSR-0 classes), infrastructure for building web services, dependency injection, and so on.

We then did a round of voting where we could each choose 3 of those things in order to try to determine which of those were the most important.

Two things became instantly clear during this exercise:

1. The items encompassed under WSCCI really spanned at least 3 separate major areas: Web Services, more robust ESI-based layouts (think Panels only more powerful), and cleaning up our underlying toolset to be a more loosely-coupled framework.
2. The underlying architecture to support RESTful calls to Drupal that makes all of the other things possible was deemed the most important thing to focus on.

Scope resolution

After a good chunk of discussions, all were in agreement to scale back the scope of the initiative to just the "Web Services" piece, and spin off the Layout/blocks/related-UI parts to a separate effort.

Furthermore, some efforts, such as PSR-0 and Unified Plugin system, were only semi-related to the WSCCI initiative in the first place, and just happened to become relevant for it. Work on those efforts will continue as part of the general Framework community efforts.

Architecture

Fabien was able to offer a tremendous number of insights as to how various Symfony2 components could help provide underlying structure for Drupal core to support Web Services out of the box. Essentially, most of what the WSCCI team had been trying to work toward, in the abstract, was already implemented within Symfony2. While some implementation details were different than what we had in mind, the end result is almost exactly what we have been trying to accomplish. We therefore agreed that the best way forward was to leverage several Symfony2 components within Drupal as a way to speed progress toward that end.

Benefits

Some of the concrete benefits that would come out of these changes are

• An underlying framework that is a first-class REST citizen. That means developing web services becomes easier and more efficient, because the plumbing is already in place. An HTML page is a particular case of a REST service.
• Because the core system will be fully REST-ified, we'll be able to improve existing APIs and expose Drupal content in a natively RESTful way. For example, our current AJAX system doesn't comply with REST standards, but with these changes, can be cleaned up to do so.
• That in turn makes Drupal-to-Drupal communication, content staging, content sharing, and a host of other related tasks easier.
• The use of widely used libraries and techniques makes Drupal more approachable to new developers.

Why does this matter?

As it has evolved into an increasingly powerful system, Drupal has gotten increasingly complex and is not as easy to start developing with as it once was. Many developers are nervous about continuing that trend. Managing complexity is a challenge faced by many software projects, and one approach is to use standardized patterns and components.

Due to its long support for PHP 4, as well as some unique needs, Drupal does not take full advantage of standardized patterns and components. The complexity of the custom code that’s used and the non-standard architecture combines to create a barrier to entry for developers new to Drupal (both experienced and novice developers alike).

Meanwhile, the web is constantly changing around us. Web services and mobile are more important than ever, and with that comes the need to have more flexible page and layout capabilities. Now feels like the right time to modernize Drupal’s capabilities to bring it to the forefront of modern PHP systems and web systems in general.

While changing Drupal's core plumbing to address these needs, it's also a good opportunity to do so using more widely understood and modern techniques and libraries. Leveraging the work of a fellow open source community lets us get a jump on these changes to build a more powerful, more flexible, and more easily learnable system in less time than it would take to go it our own.

While these changes may seem large, and some of them are, we believe that they will achieve the right balance between empowering Drupal's design and architecture, and moving toward more modern, standard, well-tested code and techniques to empower a new generation of developers. I hope you are as excited as we are!

Draft Redbook, last updated: Thu, 16 Feb 2012

- Read about best practices learned from the field
- Learn about SVC performance advantages
- Fine-tune your SVC

This IBM® Redbooks® publication captures several of the best practices based on field experience and describes the performance gains that can be achieved by implementing the IBM System Storage® SAN Volume Controller at the 6.2 level.