Skip to content

What is Quink?

2014 March 25

logo-blue-high-rezQuink is an extensible, end-user friendly in-page WYSIWYG HTML editor, designed for mobile first.

It allows developers to add rich input and self-edited areas into web pages and web apps.

You can find it on www.quink.mobi and github.

Why did it get built?

The trigger was that I could not find a good solution for editing rich content on mobile, specifically on the iPad platform – at least not one that hit all the key points I was aiming for. We (IMD Business School) have had an iPad app in production for supporting our course participants since June 2010, and I wanted to move beyond plain text annotations on PDFs, notes and plain text discussion forums with file attachments. I wanted participants in our courses to be able to create richer content for themselves, for sharing with fellow participants, and for feedback to and from the professors.

Once started down that road, there are a host of decisions to make, top of the list being:

  • data / file format
  • editing capabilities
  • separate app or in-app component
  • openness to variety of use cases, or focus on a tight scope

There is more detail on these points below, but after initial consideration, I was looking for

  • an HTML editor that works within the browser
  • with a good UI/UX for basic rich text editing,
  • that could be embedded in our app, and
  • which had a good API and plug-in architecture to allow great flexibility.

So I started looking for solutions. While there are many things that fulfill some of the requirements, I simply could not find one that would work well on the iPad, the primary target. In the end it seemed to come down to a choice between doing significant work on somebody else’s architecture, all of which were designed for the desktop browser, or starting from scratch and focusing primarily on the mobile environment and our own needs. Even that decision is not a no-brainer. But the end result was that we followed the path that led to Quink.

Architecture

I won’t go into all the alternatives considered here, but for me the decisions were reasonably clear:

Data format: HTML

For me, HTML is the only rational solution for the base format. It is the most versatile and widely used document format, has a really excellent track record of backward compatibility, and is free of proprietary control. If it seems a little strange, I guess that is just because people don’t think of it as a document format in the same category as pdf, word docs, and so on. The only thing it is really tricky to do is precise and locked-down control of presentation and layout. But in the multi-device world, I see that less as a critical feature than as a lurking problem. I would argue that the natural tendency of HTML to flow is more valuable, though more difficult to work with.

Separate app or in-app component.

The easy option would be to break out into specialist apps / editors. That makes extensibility simple, but it provides a really horrible user experience for many use cases and is simply unusable for some core requirements. Using 3rd party apps also creates all kinds of problems about cross-platform requirements, compatibility, and data management. For  something as core as document creation and viewing, we needed something we could build into our app and our web portals, and be sure it would just work.

Editing Capabilities

The minimum requirement was easy creation of ‘rich text’: some basic formatting for headings, lists, emphasis. The ability to include images. That’s really the core requirement, and covers 80% of immediate uses. But there is always the remaining 20%…

Tight focus or broad applicability

Beyond the basic capabilities, I knew from the start that there are a million things that will be required at some point: tables, graphs, vector graphics, video, audio, and just about anything you can imagine. When each of these will become important or critical is unknown – so the key requirement is to have something that is extensible in response to new demands and use cases. HTML and the web stack provide a good framework for allowing this, and for using components developed by others – both proprietary or open source. So our goal was to create an architecture where we could employ specialist content editors developed by others, out of the box. I always strive to create architectures that put as few limits as possible on the future without incurring unreasonable current costs – and I think we have achieved that.

Design – What is special?

Content divs and tagging

Probably the most significant thing about Quink is the approach to extensibility and plugins. The idea is that a page is made up of units, for each of which you may need a specialist editor. Quink exploits the HTML structure where a page is made up of a set of divs and elements. Divs provide clean boundaries for content. Some divs may be tagged to identify the editor functionality that is appropriate for editing them. If the user wishes to edit such tagged content, then the specialist editor is loaded, passed the content that it needs to edit, and the Quink core steps back to let it do its job. When it is finished, the modified content is updated in the HTML.

The base implementation of this was designed to allow the use of editors that have no knowledge of Quink. There are very, very few requirements that an editor must satisfy to allow it to be used as a plugin, and the system uses adapters so that the requirements on the underlying component are functional capabilities, not any specific API. To be eligible as a plugin, some editor need only:

  • Be loadable in a web page
  • To have some method of delivering the edited content to the Quink core so that it can be dropped into a page – ie renderable HTML.
  • In order to be re-editable, it also needs a means of having the HTML sent to it.

Being loadable includes being hosted on different servers. Quink defaults to opening plugins in iFrames, and only loading them when asked for by the user. They don’t have to be part of the root site, so you dont actually need access to source, or to own the editor component. Of course, when setting up a plugin you should trust the provider and code enough to give them access to an iframe in your page!

The mechanisms for transferring the data can include all kinds of back-end tricks if needed, though we haven’t gone down that road ourselves yet. Quink supplies the user with a button to save and exit, or simply quit the plugin – which calls a function on to the plugin’s adapter, so the plugin editor itself does not need to know it is operating inside Quink or to emit any events.

If an editor is not capable of re-editing existing content it will not break anything either, though of course it may not meet user expectations.

Adding Quink to a page & configuration

Quink bootstraps from a small script which exists mainly to set up the urls to load from and kick Require.js into action. The default bootstrap script will do the job for most installations; the target page then only needs to have a one-line inclusion of the bootstrap script, and to declare one or more divs to be contenteditable; Quink is enabled as the editor for all of them by default.

Various aspects of Quink are easily configurable. A configuration can be set up by adjusting JSON files: the plugins, toolbars, and the keymap for keyboard-driven edit functions. One of the items on the roadmap is to allow these configuration structures to be cleanly manipulated after loading.

There are also a number of things which we have found it useful to allow Quink to pull from the page query parameters: the autosave frequency and destination, the destination for an explicit POST of the content as an alternate save mechanism, whether the toolbar should pop up on page load. This approach allows referring links to change aspects of the configuration which turn out to change more frequently than it is practical to change code, and seems to be a useful pattern. In future I think this will be extended and also made more generic – and capable of being disabled.

Keyboard mapping

One of the other things which is a little unusual in WYSIWYG HTML editors, that we have included in Quink is keyboard commands. This was also driven by frustration as fairly heavy iOS users with the touch-based cursor positioning and selection. In my view this is one area where the Android UI is just miles better; but even then, trying to position the cursor and select text to replace, delete or format is slow and relatively tricky, because fine positioning is just inherently more difficult with a touch interface than with a mouse – and I am speaking as someone who has quite steady hands. From my past in mobile surveying and mapping, I know that quite a high percentage of the population have really quite shaky hands and find fine positioning on touch screens REALLY hard.

So we added an approach to allowing keyboard commands: the minimal target was simply keyboard-based navigation and selection, but the architecture delivered the ability to map a key sequence to any of the toolbar or internal commands. Because of the limitations of on-screen keyboards, we had to deliver this without control keys, and the best solution seemed to be to use a standard QWERTY key in some way. Following that line led pretty inevitably to requiring a command mode and an insert mode like Vi. This is really deeply ironic since I grew up as an emacs fan, and avoided Vi as much as I could, and now found myself forced to implement and learn vi-like sequences to achieve what I wanted.

Where we have ended up is with two ‘q’ keys in quick succession to enter command mode, and a single ‘q’ to return to normal, or ‘insert’ mode. The default map is not the same as vi, because of course many commands are more about formatting than rapid editing of plain text, but diehards can adjust to suit their tastes! To handle the limited set of keys, it is possible to set up command groups with ‘prefix’ keys, so for example we use ‘f’ for font formatting, so ‘fb’ means ‘format:toggle-bold’ and ‘fbi’  means ‘format:(toggle-bold, toggle-italic)’.

What next?

I have a long list of to-do’s.

Some should be relatively simple, such as adding a few more plug-ins; I particularly fancy the image editor Muro, not only because it seems really good, but because it is a hosted component that has the required functions, so it is also an interesting test case for the plugin architecture. After that, the next class of plugin to work on is grid/table support.

Good support of Android devices is certainly high on the list.

After that, there is some significant work to be done on div and css style management. Right now, Quink just exposes the browser behaviour for these areas, which is limited and often rather flaky. In principle this is all do-able, but doing it well with a clean architecture is an interesting challenge.

We have some other cool ideas, but they are in a phase of stealth mode experimentation just now.

Open Source

We have released Quink under the Gnu Lesser GPL. The aim is to find a good balance between maximising the usefulness and user base around Quink (by being relatively liberal), and to get help and input from the community on improvements. Our current understanding of the lesser GPL with regard to Quink is that it allows people to use Quink in their apps and sites, or write add-ons and plugins without being obliged to open source everything – thereby maximising its usefulness. However, if there are bug fixes or compatibility fixes that people find and make to the core, it is the least they can do to publish them. It would be great if people become proactive and contribute/publish plugins, plugin adapters or other significant enhancements, but that is entirely voluntary.

IainCookeTIain is CTO at IMD business school in Lausanne. He led IMD’s pioneering use of iPads in education as part of a longer term re-engineering of IMD’s operational and learning support systems.

Prior to joining IMD in 2008, Iain has worked in a variety of industries, but always at the forefront of technology development and disruptive change. In the early 90s, Iain co-founded a software start-up focused on mobile pen computing and geospatial solutions. The solution that he created enabled Ordnance Survey to be the world’s first mapping agency to use a 100% digital map collection and production system, and helped revolutionise the industry of creating and consuming geographic information.

 

 

 

Lack of women speakers at conferences

2014 March 11

A couple of weeks ago I asked User Group Leaders why there are so few women speakers at conferences. I received some comments that I would like to share with you.

First I got the statements, reasons why women do not attend/speak at conferences

  • “I’m an ecology/evolution person so our conferences are more gender balanced than elsewhere. In fact we tend towards more female speakers than male speakers in general sessions because there are more female PhD students (60:40). However, there is a big issue with symposium speakers [, …] where speakers have to be selected and invited to speak.

I’ve been organising a symposium this year in the area of computational evolutionary biology and I also run our seminar series at Trinity. I have found it very hard to get female speakers compared to male speakers. Firstly it seems that some fields genuinely do have fewer women. Secondly[,] even in fields where there are lots of women, unconscious bias leads people to suggest male speakers first (I do this too but now force myself to think of equal numbers). All this is before even inviting people! When I do ask women I find that they are much more likely to say no due to a) being busy and not wanting to travel so much b) childcare issues and c) not really seeing the benefit to their career. I also think there is a bit of impostor syndrome going on here where women think they have nothing to say, or are being invited as the “token” woman.

I’ve also found a really weird thing that happens with PhD students of theoretical or computational professors – all the male students do method development, programming etc and all the female students do empirical studies. I myself am more of a “high end methods user” than a competent programmer though I’m trying to improve! …” – Natalie, Ireland

  • “I think the cause of there being so few women presenting at conferences is systematic bias– no single person is consciously discriminating, the system as a whole is. So, for example, if people are looking for “the best” JavaScript developer to present at a conference, the systemic bias makes that person much more likely to be a man.” – Jonathan, UK
  • “From my experience as a student and as a PhD candidate, whenever I was participating together with my colleagues at different conferences or competitions I was the one doing the presentations. […] [A]s you correctly noticed there are very few women in IT Conferences. Two weeks ago I attended a conference in Barcelona and there was only one woman speaker.”  – Andreea, Romania
  • “Unfortunately we do not have girls in our user group. It seems girls and women simply are not interested in listening to our topics. I think it may be counted on the fingers of one hand the number of female presence in our pluriennal activity. If you get an answer to this question I’m the first that would like to know.” – Andrea, Italy
  • “I would love to know that too. I’m not the leader of AIRO anymore (where we had 3 women as active members, but they were not speakers), but I’m the leader of Alagoas Linux DevGroup and there [are] no [women] in this group.” – Arthur, Brazil/Romania
  • Groups (from the inside): some of us who have tried to drive IT user groups have worked on this, trying to address it both in the group and its meetings. But, essentially, we have failed. Even Scandanavia seemed no different, at least at the turn of the millennium.  This often total lack of participation contrasted with many of my earlier (70s-90s) experiences in various other groups.

I see the only way to beat the blockage would be to have minimal representation — say, 30% — of either sex; all groups, programme committees, etc. should attend to that.  That may seem harsh but hardly more so than the other aspirations of any group or event.

Is there anything to learn from foreign-language schools where often over 75% are female?” – Charles, UK/Spain

  • “It’s a pity that so few women are speaking at conferences. More often than not they have more to add than they might think.” – Daniel, The Netherlands
  • “In the 3 years of London Web Performance we’ve never had a female speaker either (although we’ve had some fantastic contributions from female members at WebPerfDays!).” – Stephen, UK
  • “In my career as developer/devops I worked in many companies, but I have to say few women were/are “hands on”, they are normally in product and project management.

Thinking about the university – I studied Computer Science – women were probably around 1 in 10, and very few ended up programming. Most of them went to teaching or doing Project Management. … The fact they feel a minority probably makes them quiet I’d say, and possibly fearing the audience? – Mauro, UK

  • “Our female turnout for meetups would be 1 in 50 maybe? I wouldn’t say how many are female in the total Meetup group (1300 registered) as it’s not tracked.

I agree that the proportion of females in sysadmin is very low, although perhaps slightly higher % of females DBA’s (maybe).

Developers the % of women is increasing, and the % in QA is quite high.“ – Stephen , UK

  • “I would think that the lack of female speakers is due to the fact that there is a lack of females in IT as a general. We are still rare. […] There have been conferences where I was the only woman attendee. Another thing as well is blogs, I personally know only of about 10 IT blogs written by women as opposed to the hundreds out there written by men. Speaking at conferences and public places is not for everybody … and requires guts and a good topic. It definitely is not the lack of knowledge, but the lack of confidence that your topic will be interesting enough […] to listen to.

 [This] is [something]I think about quite often. You cannot be asking about the lack of females in IT[:] it is a much bigger problem – where are the women in science, [boardrooms] or higher positions in the work hierarchy? There are just the few and they even feel the need to write books about it!. [So] definitely the problem is much deeper, it comes from the fact that women still feel […]  [they’re] working in a [male]-dominated area is an act of defiance while that should the most natural thing in the world, but years of staying at home to look after the household made us […] unsuitable for the new age where women are equal with men in terms of what is expected of them if not more.” - Kalina, UK

  • “I feel ladies are not speaking because they are not always asked to! Yes, there may be fewer women in the industry than men, but successful and phenomenal women DO exist and they are willing to make a valid contribution!

Conference organisers should make the decision to increase the percentage of women speakers (based on merit and not gender of course!)” – Chisenga, Zambia

  • “The funny thing is that our meetup has only 3 women out of 67 members! Amazing, isn’t it? I’ve attended also some conferences last year and I noticed the same [as] you. Very few women speak at conferences. My explanation about this is that in our world (computer science, software development, etc.), the majority of the professionals are indeed men – women don’t love this kind of professions but I don’t understand why – so it’s normal to see more men. However, I strongly believe that women are much better to speak at conferences[.]They have a different speaking style and I always prefer to attend a presentation by [a] woman rather than [a] man.” – Patroklos, Greece
  • “According to me, it’s [female] culture. [They] think that [they’re] not able to do all [the] things that a man can do, including in technology fields.” – Yvan, Cameroon

 

Then I got the blogs I should be reading, the groups I should be attending  – it might be a good idea for all of us to read these blogs or attend these meetings/groups.

  •  “In the geek feminism blog you can find some posts that talk about why there are less women giving talks at conferences.

The reason why the Code of Conduct [is] so popular now in conference[s] is thanks to the work of the Ada Initiative. See https://adainitiative.org/what-we-do/conference-policies/ And http://adainitiative.org/2014/02/howto-design-a-code-of-conduct-for-your-community/” – Ana, Spain

  • […] we have launched an initiative to highlight this very issue called ‘Women in IT’ which was born from sessions held at UKOUG Apps and Tech conferences last year.

Specifically we are looking for women who will speak at Special Interest Groups […], act as a mentor to other women in IT, link with BCS’s Computers in Schools initiative , and ask women in IT to write a short piece about their experience. The key objective for this initiative is to act as a positive catalyst for change, to support and encourage women in IT.” – UK Oracle User Group, UK

  • Have you seen this paper? It discusses the issues and data.” – Natalie, Ireland

 

And now my views

Reading through these comments it feels that men are baffled by the lack of women in their groups and women do not join because they are not asked to.

Who talks more?

Contrary to popular belief, we know that women do not talk as much as men in a mixed group (see Coates, Jennifer (1993) Women, Men and Languages, 2nd Edn, London: Longman). Women’s talks are often referred to as gossip, chatter, nag, rabbit, yak and natter. You must admit that this will not encourage women to talk at User Group Meetings or conferences. Not only do we believe that women talk too much when research shows that men on average talk more than women, [this] also indicates how women and women’s activities have tended to be undervalued.

Conclusion: Women do not feel comfortable speaking to a mixed audience as they feel pressured by men through centuries of being told that their conversation is rather shallow.

Interruption and dominance

It also appears that men will interrupt a woman far more that they will interrupt another man. These findings seem to show that men act as if they have more right than women to speak in mixed-sex conversations.

Conclusion: Women are used to be interrupted – so why bother?

Use of diluting phrases

According to Lakoff (Lakoff, Robin (1975) Language and Woman’s Place, New York: Harper & Row) women use linguistic forms that dilute assertion – sort of, like, I think, kind of showing that women are less confident than man and feel nervous about asserting anything too strongly. Other studies claim that women prefer to avoid conflict and so use forms which, by being less direct, allow disagreement to take place without confrontation.

Conclusion: Women are afraid to upset their audience by stating clearly their opinion.

I found some of the comments a little too patronising – I am a woman after all. I found other comments a little too close to the truth – women do not go to meetings because they do not have the time due to childcare, housework, family care. Should we then look at our upbringing and see where our society is going wrong?

  • baby boys in blue, baby girls in pink
  •  jewellery for baby girls
  • young girls being told off for being boisterous when little boys are praised for being competitive and pushy

Our society has role cast women – is that why they do not speak at conferences?

Game time

Thank you Stephen, London, for pointing this out.

Conference Speaker Bingo: a bingo card full of excuses for not having more female speakers at STEM conferences

—-

PS. Some of my research was taken from Thomas, Linda et al, (2004) Language, Society and Power, 2nd Edn, Abingdon: Routledge

PPS. Just learn that Debian are organizing MiniDebConf Barcelona 2014 –  where everybody is invited but talks will be by women only.

Other resources

  • Tech is too important to be left to men!’, Europa, 6 March 2014 – A European Commission campaign to celebrate women in ICT and inspire young women to get involved. Includes some interesting (if perhaps worrying) statistics
  •  ‘Tokenism’, Geek Feminism Wiki - Overview of ideas of tokenism at conferences
  •  Dudman, Jane, ‘Five myths about why there aren’t more women at the top’, The Guardian, 8 March 2014 – Points 2 and 3 are particularly worth reading: point 2 mentions unconscious bias and 3 dissects the argument that able, clever women are already visible, so the women who don’t make it just haven’t tried hard enough
  •  Gahran, Amy, ‘Women at Tech Conferences: Look Beyond Tokenism (comment to Scoble)’, Contentious.com, 2 August 2005 – Another blog discussing tokenism, and dissecting the argument that there is no bias involved in choosing speakers – they are purely chosen on ‘merit’
  •  Kantor, Jodi, ‘Harvard Business School Case Study: Gender Equity’, The New York Times, 7 September 2013 [BEHIND A PAYWALL] – Like ICT, Business Studies is another field with few visible women. This article explains the changes Harvard Business School made to encourage a better learning environment for its female students – not only was there a gap in grade achievement, but female students weren’t speaking or being nominated for awards. This not only covers the method but also the result – it was extremely successful but some male students weren’t happy with the experience
  •  Urbina, Michael, ‘101 Everyday Ways for Men to Be Allies to Women’, Michael Urbina, 26 July 2013 – A lot of male commenters write about how they wish the situation was different. This is a list by a male feminist of the ways men can be more aware of the effects of patriarchy and how they can help
  •  Williams, Zoe, ‘Why female techies in the 21st century face a stone age work culture’, The Guardian, 7 March 2014 – Overview of the problem of the shortage of women in ICT, possible causes
  • Henson, Val, “HOWTO Encourage Women in Linux” – This article is 12 years old but unfortunately it could have been written yesterday

National Hack the Government 2014 – 8/9 March

2014 March 3
by Josette Garcia

National Hack the gov logoNational Hack the Government is an annual HackCamp run by Rewired State. As well as bringing great people together to have fun hacking and building things, we hope to improve transparency, open data and produce demo-able ideas that can be followed up and put into practice in real life. This is done by holding a competitive event for creating prototypes and building ingenious (and occasionally tongue-in-cheek) projects that help improve local and national services and make use of open data.

Now in its 6th year, an even bigger event has been put together to host local communities hacking around local data, issues and problems bringing in the National part in the title to life. A bar-camp has also been added on the Saturday to help share information and insight – to organically define some of the challenges that will be hacked on. This explains the title “Hack-Camp”!

The event will be followed up by a show-and-tell of the finalists’ work – taking the winners from each centre and allowing them to demo their ideas to the government, businesses and the wider community.

National Hack the Government is sponsored by:

 

sponsors issue 3

 

 

 

 

 

 

 

 

 

 

 

 

 

 

FutureGov is excited to be involved with National Hack the Government by hosting local communities’ challenges on Simpl Challenges, including GlasgowExeterLeeds and Bournemouth.

Simpl Challenges is our innovation platform that connects local public services and organisations with innovators and ideas. Gathering ideas before the event on Simpl will give more time for learning, talking and listening to each other, and building some fantastic prototypes on the day itself.

Follow the links below to find out more and submit your ideas for your local event, or you can take part remotely by submitting your ideas to the UK section:

You can also comment on the ideas submitted, so that the idea owners can get valuable feedback on their projects before the event has even started.

rewired_stateRewired State creates bespoke hack events that bring creative developers, designers and industry experts together to solve real world problems – promoting and supporting more than 1 200 of the UK’s most talented and inventive software developers and designers, as well as nurturing 1 500 promising world-wide developers under 18 through the Young Rewired State network.

 

 

 

London PHP Conference

2014 February 25

HeaderHeadThe 9th Annual PHP UK conference is over! Lots of people, a very posh venue, great food and very efficient and friendly organizers: Johanna CherrySam Bell and Ciarán Rooney. What else do you need at the London PHP conference? Possibly some talks! I am told they were very good.

The conference was divided into 3 tracks, 27 world-wide known speakers and of course some beer events at the local pubs and Friday evening gathering in the exhibition hall.

For key notes we had:

  • What Makes Technology Work by Juozas ‘Joe’ Kaziukenas – compared making wooden chairs to building apps
  • The Future of PHP is in the Cloud by Glen Campbell – reviewed some of the key developments in PHP over the last few years and outlined how PHP can keep pace with the explosive growth of the cloud.

Some interesting talks included :

  • PHP in Space by Derick Rethans – how PHP can be used for all kinds of terrestrial and non-terrestrial purposes … Expect trigonometry and other maths, and rocket science/explosions! It feels that it should have been great fun to attend this talk.

PHP web services

  • Debugging HTTP by Lorna Mitchell –  Curl, Wireshark and Charles, the tools you will want to have at hand. Most of the comments are “great talk, will be using Charles…”. I have a very special relationship with Lorna as she helps me marketing her book PHP Web Services to her audience. Well done Lorna!
  • PHP at the Firehose Scale by Stuart Herbert – Here’s one to give the PHP bashers a well-deserved black eye! Twitter is one of the world’s best know social media sites, handling over 500 million public tweets a day (that’s around 6,000 tweets a second). How do they do it? With the help of PHP of course.

Other activities:

  • If suffering from an information overdose and wanted to relax, you could then go to THE CLOUD BAR (Presented by Engine Yard) – for an awesome chill out space, free swag, free coffee, and free info about everything cloud.
  • Or if feeling full of energy, you could join the HACKATHON (Presented by JetBrains) – alongside the main tracks and late into Friday night using Sochi Winter Olympics API data. Prizes were awarded throughout the Hackathon for the most innovative hacks including two free tickets to next year’s conference, PHP Storm licenses and more.

Make a note in your diary for the 10th London PHP Conference which I am sure will be even more awesome.

FOSDEM 2014

2014 February 7

fosdemThe two-day long biggest Open Source meeting in Europe is over! As you know, FOSDEM is a free event that offers open source communities a place to meet, share ideas and collaborate. It is renowned for being highly developer-oriented and brings together 5000+ geeks from all over the world.

O’Reilly has sponsored FOSDEM since 2002 when it changed its name from OSDEM to FOSDEM. So since 2002, I have been attending FOSDEM – not attending the talks but selling numerous O’Reilly books with my colleagues. Our table(s) is still in the H Block but the location has improved, we are now a few metres away from the doors which means it is no longer so cold. Even though we are very well looked after by the members of the organisation and by the delegates, providing us with tea and coffee, we still cannot have lunch until 4 pm as we are so busy. Please note: this is not a complaint just a fact. Thank you guys for bringing us food and drinks.

For the last couple of years, I started the weekend by going to the Delirium to meet some friends. The Delirium is a huge pub, two minutes from the Grand Place. I believe most of the drinks are sponsored so you can imagine the amount of people going there – it is not big enough to cater for everybody so lots of people are drinking, talking, greeting each other in the street. After an hour or less and a long wait for a coke, I had to leave as I was already thinking of getting up early to set up for the next day. This year we were incredibly lucky. Our 51 boxes of books were at the other end of the hall – not good. Our luck changed with the arrival of 3 guys coming from South Germany – unfortunately I do not know their names nor even the town they came from.  At around 7 am  on Saturday morning they came in the hall and said something like “it is cold outside and it is raining, can we stay in?” “But of course, no problem,” was the answer, “but you might have to help us with these boxes.” And those 3 great guys put the boxes on a very dilapidated trolley (several trips), brought the boxes and opened them for us. To me these three gentlemen epitomise FOSDEM – helping each other. For some of you FOSDEM is the meeting of great open source minds, for me it is the meeting of friends – some I see only once a year, some more often but always very happy to see each other. There I learn about your achievements, your dreams (one moved to Facebook in San Francisco, another one had a baby and so on).  One man very proudly showed me one of the first O’Reilly/FOSDEM bags that we created years ago. With a lot of pride and very carefully, he got the bag out of his pocket and filled it with new books.

I will not bore you with a description of the content of FOSDEM, you can see that on Philip’s (Fosdem.org) video below.

Saturday evening we had dinner with the Perl mongers – again a very multinational gathering. As the dinner was for 50 people, I will only mention a few –  Liz and Wendy, the famous Dutch duo who always sponsor these dinners; Ovid Poe, O’Reilly author; Laurent Boivins and Marc Chantreux, Perl France; Sawyer X who I hope will be interviewed soon about Dancer and published on this blog; Marian Marinov, from Sofia, who is organizing YAPC::Europe 2014 (22nd-24th August). I think I only talked about YAPC and how to make it an even greater conference.

Two of our authors came to see us:

zero mqreal world ocamlPieter Hintjens, author of Zero MQ

Anil Madhavapeddy, co-author of Real World OCaml

Pieter and Anil, should you be reading this post, I can confirm that we sold all the copies of your books – thank you for the marketing assistance during your talks.

 

I met Sarah Novotny, co-chair of OSCON for the last couple of years – unfortunately being at FOSDEM, we did not have a chance to talk apart from a very brief greeting and see you later. I was also extremely happy to meet Constantin Dabro who is the leader of the Burkina Faso Java User Group.

On Sunday evening, after packing 7 boxes of leftover books, somebody told me that I was the mother of FOSDEM, then I thought I am not the mother but I feel like great-grandmother or somebody who had run a couple of marathons in two days :))

Life as an Artist && Geek

2014 January 27
by Bart McLeod


There is no typo in the title of this post. It intentionally reads ‘&&’. && is a boolean operator used in many programming and scripting languages. Why would an artist care about a boolean operator? That’s correct: because this particular artist is also a geek. “What’s a geek?”, other artists might ask. A geek is someone interested in tech (programming, devices, electronics), but also interested in social interaction.

The artist part in the title refers to the fact that I paint and sculpt, and the geek part is about my interest in php and ZendFramework and a few other things one can program with and for the web (like 3D scenes). To make things worse, I also like to write. Why worse? You guessed right again: it leaves me craving for time.

Can I really be an artist and a geek (and a writer) at the same time? Or will any or both suffer from the fact that a day has only a bare twenty-four hours? That’s where the boolean operator comes into play. It’s up to you: if you think I’m really an artist and I’m also really a geek, then both operands (Artist and Geek) evaluate to true (as geeks tend to say) and the entire expression will also evaluate to true, the entire expression meaning ‘Artist && Geek’.

Leaving it up to others if I’m either of the two saves me a lot of headaches and hopefully some time. So although I definitely think of myself as Artist && Geek, I shall not be bothered whether this title is true or false.

With that out of the way, I should come to answering Josette’s question that led to this post: what’s it like to be both?

Being an Artist and a Geek

The fact that I seem to be always fighting for time is not the most interesting part. A lot of people do and some even go on training courses to learn how to manage their time. I don’t. Why not? Because losing some time every now and then is part of being an artist. In general, it is part of being creative. What happens when you lose time? You lose time either because you are not paying attention to it, or because someone else is wasting it for you. Wasting? Not paying attention? Let me look into those two a bit further.

Not paying attention

Not paying attention reminds me of my teacher in second grade. She shouted at me when I sat in her classroom daydreaming. Is daydreaming a waste of time? Daydreaming is what creative people do to get ideas. Nowadays, I do not get enough time to daydream, because I have a day job as a self-employed geek and also because I’m addicted to typing code and watch it come to life, so I tend to code and write about coding during as many evenings as I can steal from the remainders of my social life. Whenever I do get a little time to daydream, it tends to be on the drive to work when the sun is rising from the early morning fog:

A few milliseconds of distraction on my way to work

It is dangerous to daydream in the car while driving, so the dream only lasts long enough to capture a picture (a few milliseconds are sufficient) that I can paint later, on one of the rare days that I can spend in my art studio. I see other artists develop really creative and great ideas and I can tell that they spent a lot of time dreaming. I’m not a violent person, but it’s better that my second grade teacher and I do not ever meet again.

Wasting time

Can time really be wasted? If in this busy life, you have to wait for your dentist for half an hour because he needs a little extra time to help a new customer, is your time wasted? What would the Artist do with that time? He would look at the other people waiting. He would imagine what kind of lives they live, maybe even speak to them to find out. He would draw a sketch of the bored and waiting and later turn it into a painting. But what does the Geek do? He is prepared for this kind of situation: he takes out his MacBook Air that he can carry everywhere because it weighs next to nothing. And he starts coding, because he’s an addict, but also because he knows that since there is no pressure, he might get a better idea than when facing a tight deadline at work. One could state that at this point the Geek really gets in the way of the Artist. On the other hand, the Artist will help the Geek to be creative, since coding is considered art or poetry by many. Thinking about it, the Geek should have remembered to bring his iPad as well so that the Artist could draw on it.

The Artist and the Geek helping each other

While the above may seem unfair in that the Geek gets in the way of the Artist and the Artist is helping the Geek at typing code poetry, the Geek is also helping the Artist by supporting him financially. The first thing people want to know about an artist is how he gets by money-wise. In my case, the Geek is allowing the Artist to do anything he likes, that is, as long as time permits (which is, like we saw earlier, not very long). But spending a lot of time on a work of art is not necessarily making it better. The conception and execution of a painting may germinate for months or even years, or a painting can be born from being thrown at the canvas in an outburst of creativity after pressure-cooking in his mind for a considerable amount of time. All of this can be done independently. There is no need to sell any of these artworks, although most of them are for sale and many will eventually find a new owner.

Painting delivered using Test Driven Development over a period of four years with a few milestones along the way

For the above painting I did four different small sketches in different types of paint, only to figure out what should be on it and what would better be left out. It took me four years to decide.

Today’s art

There is something about today’s art that makes me question it a lot if it is done in a traditional way. This includes my own art. The famous artists from the past are admired because they found the most powerful ways available in their era to express themselves, or to express an idea of general interest. They also found new ways of expression. Some great artists were entrepreneurs, some successful, some not so successful. I think you can compare Rubens to Stephen Spielberg. Both orchestrated large scenes, using the most powerful visualization techniques of their time. Both didn’t do this alone.

Many say that a true artist chooses the best materials to express his idea. This can be said of both Rubens and Spielberg. That’s where my first problem lies: I tend to like certain materials, while I do not focus much on ideas. I like traditional paint and wood. So I mostly paint using traditional painting techniques and I sculpt in wood. Wrong approach! Today’s most powerful material is the byte. What? Yes, the byte. The fact you’ve read this far proves the power of bytes. Bytes become even more powerful if you create games with them. Games, especially 3D games, deliver a total experience that take possession of their player. It’s the most powerful and immersive way to express ideas currently known to mankind.

Yet the old techniques have not died completely. When you write a book, the reader immerses himself in it. While this requires effort from the reader, the immersion can be complete. The same is true for paintings, but they shall not be judged from their reproductions (displayed on either a computer screen or in print). A reproduction of a painting does not allow for immersion. It is simply impossible, believe it or not. The immersive qualities of a painting are crafted by the painter by means of continuous deployment. A painter steps back and forth in front of his work to test the immersive qualities. These differ depending on the distance to the work and the way the light falls on the painting. These qualities are completely absent from any type of reproduction.

With people staring into their tablets and smartphones all day, less people have time to go out and see actual paintings, let alone immerse themselves in them. Research has proved that if they go to a museum to look at art at all, they look for 9,000 milliseconds on average (it hurts too much to write this number down expressed in seconds, or even hours, so I’ll leave that as an exercise to the reader). Therefore, I fear for the future of painting in the traditional sense of the word and I am glad that I also have some power over bytes: the most powerful raw material of the modern age.

Some encouraging thoughts

I think guaranteed ways to waste time exist. It is a waste of time to do repetitive work. You’ve done it before, it is not a new experience. It is detrimental to your creativity and brainpower. Repetitive work should be automated, either by machines or by coding.

Although bytes have a lot of power, nothing compares to recreating an atmosphere as he has experienced it in a landscape, in the opinion of the Artist, with his own hands and traditional paint. This requires vision, experience, speed, creativity and courage. His own judgement is the harshest he can get. Nothing compares to holding an idea in his hands and turning it around, knowing that he has shaped this out of wood that didn’t really want to be shaped: wood that resisted like mad, but had to give in to his insistent chiseling.

Bart McLeod working on the Angry Little Man
Before painting it

All about Catalyst – interview of Matt S. Trout (Part 3 of 3)

2014 January 27
by Nikos Vaggalis

What about the other Perl frameworks, Dancer and Mojolicious? How do they compare to Catalyst?

Dancer’s big strength is making things quick and easy for smaller apps; you don’t have to think in terms of OO unless you want to and plugins generally shove a bunch of extra keywords into your namespace that are connected to global or per-request variables. Where Catalyst doesn’t have an exact opinion about a lot of the structure of your code but very definitely insists that you pick one and implement it, Dancer basically lets you do whatever you like and not really think too much about it.

That really isn’t meant as a criticism: somewhere along the line I picked up a commit bit to Dancer as well and they’ve achieved some really good things providing something that’s as little conceptual overhead as possible for smaller apps, and something where there’s a very direct mapping between the concepts involved and what’s actually going to happen in terms of request dispatch, whereas Catalyst abstracts things more thoroughly, so there’s a trade-off there. I mean, I was saying before that empty methods with route annotations almost always end up getting some code in them eventually.If you get to 1.0 and most of those methods are still empty, you might’ve been able to write a lot less code or at least do a lot less thinking that turned out not to have been necessary if you’d used Dancer instead. Equally, I’ve seen Dancer codebases that have got complicated enough to turn into a gnarly, tangled mess and the developers are looking and thinking, “You know, maybe I was wrong about Catalyst being overkill…”

I love the accessibility of Dancer though, and the team are great guys. I’ve seen the catalyst community send people who’re clearly lost trying to scale the learning curve to use Dancer instead, and I’ve seen the Dancer community tell people they’re doing enough complicated things at once to go look at Catalyst insteadand hey, we both run on top of PSGI/Plack so you can have /admin served by Dancer and everything else by Catalyst … or the other way around … or whatever.

Meantime, Mojolicious is taking its trade-offs in a different dimension.Sebastian Riedel, the project founder, was also founder of Catalyst; he left the Catalyst project just before the start of, I think, the 5.70 release cycle, because we’d acquired a lot of users who’d bet the business on Catalyst’s stability at that point, and a lot of contributors who thought that was OK, and Sebastian got really, really frustrated at the effort involved in maintaining backwards compatibility.

So he went away and rethought everything, and Mojo has ended up having its own implementations of a lot of things, focused on where they think people’s needs in web development are going over the next few years. A company heavily using Mojo open sourced a real-time IRC web client recently, doing a lot of clever stuff, and Mojolicious helped them with that substantially. But the price they end up paying is that when you step outside the ecosystem it’s quite jarring, because the standards and conventions aren’t quite the same as the rest of modern Perl. Mojolicious has a very well-documented staged backcompat breakage policy which they stick to religiously, and for “move fast and break stuff” style application development, I think they’ve got the policy pretty much spot on and they’re reaping a lot of advantages from it

But for a system where you want to be able to ignore a section that users are happy with and then pick it up down the line when the state of the world changes and it needs extending (for example, for the people doing boring business systems where finding out 5 seconds sooner that somebody edited somethingwould be nice to have) but what really matters is that the end result is correct, then I’m not sure I’m quite so fond of the approach. I think if you were going to talk about a typical backend for each framework, you could say that Dancer’s would be straight SQL talking to MySQL, Catalyst’s would be some sort of ORM talking to PostgreSQL, and Mojolicious’ would be a client library of some sort talking to MongoDB. Everybody’s going to see some criticism of the other two implicit in each of what I said, but take it as a compliment to their favourite: if they don’t, that’s because my metaphor failed rather than anything else

I can’t recall a time when I’ve seen an app that was a reasonable example of its framework where I really thought that either of the other two would’ve worked out nearly as well for them... with the exception of a fairly small Catalyst app that, in spite of being, if anything, a bit small for Catalyst to make sense turned into a crawling horror when ported to Mojolicious but then again, when I showed that to Sebastian and asked, “am I missing something here?”, the only reply I got was some incoherent screaming, the underlying meaning being, “WHAT HAVE THEY DONE TO MY POOR FRAMEWORK?! CANNOT UNSEE, CANNOT UNSEE! So I think, as with Dancer, its a matter of there being more than one way to do it, and one or other of them is going to be more reasonable depending on the application.

What’s in Catalyst’s wish-list, in which direction is the project moving and do you think that someday Catalyst’s adoption will be so widespread that it will become the reason for restoring the P=Perl back to LAMP?

 The basic goals at the moment are a mixture of adding convenience features for situations that are common enough now to warrant them but weren’t, say, five years ago; continuing to refactor the core to enable easier and cleaner extension; and to figure out a path forwards that lets us clean up the API to push new users onto the best paths while not punishing people that use older approaches.

 As for putting the P back in LAMP? I’ve regarded it as standing for “Perl, PHP or Python” for as long as I can remember. It doesn’t seem to me that treating it as a zero sum game is actually useful: in the world of open source, crushing your enemies might be satisfying but encouraging them and then stealing all their best ideas seems like much more fun to me.

What are your thoughts on Perl 6 and given the opportunity, would you someday re-write Catalyst in it?

 I’ve spent a fair amount of time and energy over the years making sure that the people thinking hard about language design for both the Perl5 and Perl6 languages talk to each other and share ideas and experiences reasonably often, but I’m perfectly comfortable with Perl5 as my primary production language for the moment, and so long as the people who actually know what they’re doing with this stuff are paying attention I don’t feel the need to that much.

 One of the things I’m really hoping works out is the whole MoarVM plan, wherein Rakudo will end up with a solid virtual machine that was designed from the start to be able to embed libperl and thereby call back and forth between the languages. So if that plan comes off, then I don’t think you’d ever write Catalyst in Perl6 so much as you could write parts of Catalyst apps in Perl6 if you wanted to… and maybe one day there’d be something that uses features that are uniquely Perl6-like that turns out to be technologically more awesome. You can still write parts of those apps in Perl5 if it makes sense, but I don’t think looking at the two languages in the Perl family as some sort of competition is that useful. I much prefer a less dogmatic approach, similar to the saner of the people I know who are into various Lisp dialects.

So it’s more about experimenting in similar spaces and learning and sharing things – and while being a language family is often cited as a reason why Lisp never took over the world … Perl taking over the world got us Matt’s Script Archive and a generation of programmers who thought the language was called PERL and fit only for generating write-only line noise whereas being a language family seems to have pretty effectively given Lisp immortality, albeit a not-entirely-mainstream sort of immortality.

 I think, over a long enough timeline, I could pretty much live with that (absent a singularity or something I’ll probably be dead in about the number of years that Lisp has existed), and I think if there is a singularity then programming languages afterwards won’t look anything like they do now… although, admittedly, I still wouldn’t be surprised if my favorite of whatever they do look like was designed by Larry Wall.

nikosNikos Vaggalis has a BSc in Computer Science and a MSc in Interactive Multimedia. He works as a Database Developer with Linux and Ingres, and programms in both Perl and C#. He is interested in anything related to the RDBMS and loves crafting complex SQL queries for generating reports. As a journalist, he writes articles, conducts interviews and reviews technical IT books

All about Catalyst – interview of Matt S. Trout (Part 2 of 3)

2014 January 20

Does all that flexibility come at a price?

The key price is that while there are common ways to do things, you’re rarely going to find One True Way to solve any given problem. It’s more likely to be “here’s half a dozen perfectly reasonable ways, which one is best probably depends on what the rest of your code looks like”, plus while there’s generally not much integration specific code involved, everything else is a little more DIY than most frameworks seem to require.

I can put together a catalyst app that does something at least vaguely interesting in a couple hours, but doing the sort of 5 minute wow moment thing that intro screencasts and marketing copy seem to aim for just doesn’t happen, and often when people first approach catalyst they tend to get a bit overwhelmed by the various features and the way you can put them together.

There’s a reflex of “this is too much, I don’t need this!”. But then a fair percentage of them come back two or three years later, have another look and go “ah, I see why I want all these features now: I’d’ve written half as much code since I thought I didn’t need all Catalyst features”. Similarly the wow moment is usually three months or six months into a project, when you realise that adding features is still going quickly because the code’s naturally shaken out into a sensible structure

So, there’s quite a bit of learning, and it’s pretty easy for it to look like overkill if you haven’t already experienced the pain involved. It’s a lot like the use strict problem writ large – declaring variables with my inappropriate scopes rather than making it up as you go along is more thinking and more effort to begin with, so it’s not always easy to get across that it’s worth it until the prospective user has had blue daemons fly out of his nose a couple of times from mistakes a more structured approach would’ve avoided.

So, it’s flexibility at the expense of a steep learning curve, but apart from that, if I could compare Catalyst to Rails, I would say that Rails tries to be more like a shepherd guiding the herd the way it thinks is the right one or the way they should go, while Catalyst allows room to move and make your own decisions. Is that a valid interpretation ?

It seems to me that Rails is very much focused on having opinions, so there’s a single obvious answer for all the common cases. Where you choose not to use a chunk of the stack, whatever replaces it is similarly a different set of opinions, whereas Catalyst definitely focuses on apps that are going to end up large enough to have enough weird corners that you’re going to end up needing to take your own choices. So Rails is significantly better at making easy things as easy as possible but Catalyst seems to do better at making hard things reasonably natural if you’re willing to sit down and think about it.

I remember talking to a really smart Rails guy over beer at a conference (possibly in Italy) and the two things I remember the most were him saying “my customers’ business logic just isn’t that complicated and Rails makes it easy to get it out of the way so I can focus on the UI”, and when I talked about some of the complexities I was dealing with, his first response was, “wait, you had HOW many tables?”.

So while they share, at least very roughly, the same sort of view of MVC, they’re optimised very differently in terms of user affordances for developers working with them. It wasn’t so long back somebody I know who’s familiar with Perl and Ruby was talking to me about a new project. I ended up saying: “Build the proof of concept with rails, and then if the logic’s complicated enough to make you want to club people to death with a baby seal, point the DBIx::Class schema introspection tool at your database and switch to Catalyst at that point”.

But surely, like Rails, Catalyst offers functionality out of the box too. What tasks does Catalyst take care of for me and which ones require manual wiring?

There’s a huge ecosystem of plugins, extensions and so forth in both cases but there’s a stylistic difference involved. Let me talk about the database side of things a sec, because I’m less likely to get the Rails-side part completely wrong.

Every Rails tutorial I’ve ever seen begins “first, you write a migration script that creates your table” … and then once you’ve got the table, your class should just pick up the right attributes because of the columns in the database, and that’s … that’s how you start, unless you want to do something non-standard (which I’m sure plenty of people do, but it’s an active deviation from the default), whereas you start a catalyst app, and your first question is “do I even want to use a database here?”

Then, assuming you do, DBIx::Class is probably a default choice, but if you’ve got a stored procedure oriented database to interface to, you probably don’t need it, and for code that’s almost all insanely complex aggregates, objects really aren’t a huge win but let’s assume you’ve gone for DBIx::Class

Now you ask yourself “do I want Perl to own the database, or SQL?” In the former case you’ll write a bunch of DBIx::Class code representing your tables, and then tell it to create them in the database; in the latter you’ll create the tables yourself and then generate the DBIx::Class code from the database. There’s not exactly an opinion of which is best: generally, I find that if a single application owns the database then letting the DBIx::Class code generate the schema is ideal, but if you’ve got a database that already exists that a bunch of other apps talk to as well, you’re probably better having the schema managed some way between the teams for all those apps and generating the DBIx::Class code from a scratch database.

Both of those are pretty much first class approaches, and y’know, if your app owns the database but you’ve already got a way of versioning the schema that works, then I don’t see why I should stop you from doing that and generating the DBIC code anyway. So it’s not so much about whether manual wiring is required for a particular task or not but how much freedom you have to pick an approach to a task, and how many decisions does that freedom require before you know how to fire off the relevant code to set things up. I mean, whether you classify code as boilerplate or not depends on whether you ever foresee wanting to change it.

So when you first create a catalyst controller, you often end up with methods that participate in dispatch – have routing information attached to them – but are completely empty of code which tends to look a little bit odd, so you often get questions from newbies of “why do I need to have a method there when it doesn’t do anything?”, but then you look at this code again when you’re getting close to feature complete, and almost all of those methods have code in them now because that’s how the logic tends to fall out.

There’s two reasons why that’s actively a good thing: first, that because there was already a method there even if it was a no-op to begin with, the fact it’s a method is a big sign saying “it’s totally OK to put code here if it makes sense”, which is a nice reminder, and makes it quite natural to structure your code in a way that flow nicely and secondly, once you figure it out in total, any other approach would involve time to declare non-method route plus time to redeclare all routes that got logic as methods and if most of your methods end up with code in them then that means that overall, for reasonably complex stuff, the Catalyst style ends up being less typing than anything else would be. But again we’re consciously paying a little bit more in terms of upfront effort as you’re starting to enable maintainability down the road

It’s easy to forget that Catalyst is not just a way of building sites, but also a big, big project in software architecture/engineering terms, built with best practices in mind.

Well, it is and it isn’t: there’s quite a lot of code in there that’s actually there to support not best practices, but not forcing people to rewrite code until they’re adding features to it, since if you’ve got six years’ worth of development and a couple hundred tables’ worth of business model, “surprise! we deleted a bunch of features you were using!” isn’t that useful, even when those features only existed because our design sucks (in hindsight, at least).

I’d say, yeah, that things like Chained dispatch and the adaptor model and the support for roles and traits pretty much everywhere enables best practices as we currently understand them. But there’s also a strong commitment to only making backwards incompatible changes when we really have to, because the more of those we make, the less likely people are to upgrade to a version of Catalyst that makes it easy to write new code in a way that sucks less (or, at least, differently).

But there’s a strong sense in the ecosystem and in the way the community tends to do things of trying to make it possible to do things as elegantly as possible even with a definition of elegant that evolves over time. So you might wish that your code from, say, 2008, looked a lot more like the code you’re writing in 2013, but they can coexist just fine until there’s a big features push on the code from 2008 and then you refactor and modernise as you and we’ve always had a bias towards modernization, so things can be done as extensions, and prioritising making it more possible to do more things as extensions, than adding things into the core.

So, for example, metacpan.org is a catalyst app using elasticsearch as a backend and people are using assorted other non-relational things and getting on just fine … and back in 2006, the usual ORM switched from Class::DBI to DBIx::Class and it wasn’t a big deal (though DBIx::Class got featureful enough that people’s attempts to obsolete it have probably resulted in more psychiatric holds than they have CPAN releases) and a while back we swapped out our own Catalyst::Engine:: system for Plack code implementing the PSGI spec, and that wasn’t horribly painful and opened up a whole extra ecosystem (system for handling HTTP environment abstraction).

Even in companies conservative enough to be still running 5.8.x Perl, most of the time you still tend to find that they’ve updated the ecosystem to reasonably recent versions, so they’re sharing the same associated toolkits as the new build code in Greenfield projects. So we try and avoid ending up too out of date without breaking existing production code gratuitously, and nudge people towards more modern patterns of use and not interfere with people who love the bleeding edge, but not force that on the users we have who don’t either. So sometimes things take longer to land than people might like. There’s a lot of stuff to understand, but if you’re thinking in terms of core business technology rather than hacking something out that you’ll rewrite entirely in a year when Google buys you or whatever, I think it’s a pretty reasonable set of trade-offs.

In the forthcoming third and last part of the interview, we talk about the the other Perl frameworks Dancer and Mojolicious, in which direction is the project moving, and whether Perl 6 is a viable option for Web development

 

nikosNikos Vaggalis has a BSc in Computer Science and a MSc in Interactive Multimedia. He works as a Database Developer with Linux and Ingres, and programms in both Perl and C#. He is interested in anything related to the RDBMS and loves crafting complex SQL queries for generating reports. As a journalist, he writes articles, conducts interviews and reviews technical IT books

All about Catalyst – interview of Matt S. Trout (Part 1 of 3)

2014 January 14
by Nikos Vaggalis

CatBot-goggles-v-02-shadowcat-200We talk to Matt S. Trout, technical team leader at consulting firm Shadowcat Systems Limited, creator of the DBIx::Class ORM and of many other CPAN modules, and of course co-maintainer of the Catalyst web framework. These are some of his activities, but for this interview we are interested in Matt’s work with Catalyst.

Our discussion turned out not to be just about Catalyst though. While discussing the virtues of the framework, we learned, in Matt’s own colourful language, what makes other popular web frameworks tick, managed to bring the consultant out of him who shared invaluable thoughts on architecting software as well as on the possibility of Perl 6 someday replacing Perl 5 for web development.

We concluded that there’s no framework that wins by knockout, but that the game’s winner will be decided on points, points given by the final judge, your needs.

So, Matt, let’s start with the basics. Catalyst is a MVC framework. What is the MVC pattern and how does Catalyst implement it?matt-screen-01-200px

The fun part about MVC is that if you go through a dozen pages about it on Google you’ll end up with at least ten different definitions. The two that are probably most worthwhile paying attention to are the original and the Rails definitions.

The original concept of MVC came out of the Xerox PARC work and was invented for Smalltalk GUIs. It posits a model which is basically data that you’re live-manipulating, a view which is responsible for rendering that, and a controller which accepts user actions.

The key thing about it was that the view knew about the model, but nothing else. The controller knew about the model and the view, while the model was treated like a mushroom – kept in the dark; the view/controller classes handled changes to the model by using the observer pattern, so an event got fired when they changed (you’ll find that angular.js, for example, works on pretty much this basis – it’s very much a direct-UI-side pattern).

Now, what Rails calls MVC (and, pretty much, Catalyst also does) is a sort of attempt to squash that into the server side at which point your view is basically the sum of the templating system you’re using plus the browser’s rendering engine, and your controller is the sum of the browser’s dispatch of links and forms and the code that handles that server side. So, server side, you end up with the controller being the receiver for the HTTP request, which picks some model data and puts it in … usually some sort of unstructured bag. In Catalyst we have a hash attached to the request context called the stash. In Rails they use the controller’s instance attributes and then you hand that unstructured bag of model objects off to a template, which then renders it – this is your server-side view.

So, the request cycle for a traditional HTML rendering Catalyst app is:

  1. the request comes in
  2. the appropriate controller is selected
  3. Catalyst calls that the controller code, performs any required alterations to the model
  4. then tells the view to render a template name with a set of data

The fun part, of course, is that for things like REST APIs you tend to think in terms of serialize/deserialize rather than event->GUI change, so at that point the controller basically becomes “the request handler” and the view part becomes pretty much vestigial, because the work to translate that data into something to display to the user is done elsewhere, usually client side JavaScript (well, assuming the client is a user facing app at all, anyway).

So, in practice, a lot of stuff isn’t exactly MVC … but there’ve been so many variants and reinterpretations of the pattern over the years that above all it means to “keep the interaction flow, the business logic, and the display separate … somehow” which is clearly a good thing, and idiomatic catalyst code tends to do so. The usual rule of thumb is “if this logic could make sense in a different UI (e.g. a CLI script or a cron job), then it probably belongs inside the domain code that your web app regards as its model”; plus “keep the templates simple, and keep their interaction with the model read-only anything clever or mutating probably belongs in the controller”.

So you basically drive to push anything non-cosmetic out of the view, and then anything non-current-UI-specific out of the controller and the end result is at least approximately MVC for some of the definitions and ends up being decently maintainable

Can you swap template engines for the view as well as, at the backend, swap DBMS’s for the model?

Access to the models and views is built atop a fairly simple IOC system – inversion of control – so basically Catalyst loads and makes available whatever models and views are provided, and then the controller will ask Catalyst for the objects it needs. So the key thing is that a single view is responsible for a view onto the application; the templating engine is an implementation detail, in effect, and there are a bunch of view base classes that mean you don’t have to worry about that, but if you had an app with a main UI and an admin UI, you might decide to keep both those UIs within the same Catalyst application and have two views that use the same templating system but a completely different set of templates/HTML style/etc.

In terms of models, if you need support from your web framework to swap database backends, you’re doing something horribly wrong. The idea is that your domain model code is just something that exposes methods that the rest of the code uses – normally it doesn’t even live in the Catalyst model/ directory. In there are adapter classes that basically bolt external code into your application.

Because the domain code shouldn’t be web-specific in the first place you have some slightly more specialised adapters – notably Catalyst::model::DBIC::Schema which makes it easier to do a bunch of clever things involving DBIx::Class – but the DBIx::Class code, which is what talks to your database for you, is outside the scope of the Catalyst app itself.

The web application should be designed as an interface to the domain model which not only makes things a lot cleaner, but means that you can test your domain model code without needing to involve Catalyst at all. Running a full HTTP request cycle just to see if a web-independent calculation is implemented correctly is a waste of time, money and perfectly good electricity!

So, Catalyst isn’t so much DBMS-independent as domain-implementation-agnostic. There are catalyst apps that don’t even have a database, that manage, for example, LDAP trees, or serve files from disk (e.g. the app for our advent calendar). The model/view instantiation stuff is useful, but the crucial advantage is cultural. It’s not so much about explicitly building for pluggability as refusing to impose requirements on the domain code, at which point you don’t actually need to implement anything specific to be able to plug in pretty much whatever code is most appropriate. Sometimes opinion is really useful. Opinion about somebody else’s business logic, on the other hand, should in my experience be left to the domain experts rather than the web architect.

Catalyst also has a RESTfull interface. How is the URI mapped to an action?

The URI mapping works the same way it always does. Basically, you have methods that are each responsible for a chunk of the URL, so for a URL like /domain/example.com/user/bob you’d have basically a method per path element: the first one sets up any domain-generic stuff and the base collection, the second pulls the domain object out of the collection, then you go from there to a collection of users for that domain and pull the specific user. Catalyst’s chained dispatch system is basically entirely oriented around the URL space, drilling down through representations/entities anyway which is a key thing to do to achieve REST, but basically a good idea in terms of URL design anyway plus, because the core stuff is all about path matching, it becomes pretty natural to handle HTTP methods last. So there’s Catalyst::Action::REST and the core method matching stuff that makes it cleaner to do that, but basically they both just save you writing:

if (<GET request>) { ... } elsif (<POST request>) { ... } etc.

Of course you can do RESTful straight HTTP+HTML UIs, although personally I’ve found that style a little contrived in places. For APIs, though, the approach really shines but basically API code is – well, it’s going to be using a serializer/deserializer pair (usually JSON these days) instead of form parsing and a view – but apart from that, the writing of the logic stuff isn’t hugely different. RESTful isn’t really about a specific interface, it’s about how you use the capabilities available. But the URI mapping and request dispatch cycle is a very rich set of capabilities – and allow a bunch of places to fiddle with dispatch during the matching process. Catalyst::Action::REST basically hijacks the part where Catalyst would normally call a method and calls a method based on the HTTP method instead; so, say, instead of user you’d have user_GET called. There’s also Catalyst::Controller::DBIC::API which can provide a fairly complete REST-style JSON API onto your DBIx::Class object graph.

So again it’s not so much that we have specific support for something, but that the features provide mechanism and then the policy/patterns you implement using those are enabled rather than dictated by the tools. I think the point I’m trying to make is that REST is about methods as verbs and about entities as first class things so it implies good URL design … but you can do good URL design and not do the rest of REST, it’s just that they caught on about the same time.

What about plugging CPAN modules in? I understand that this is another showcase of Catalyst’s extensibility. Can any module be used, or must they adhere to a public interface of some sort?

There’s very little interface; for most classes, either your own or from CPAN, Catalyst::model::Adaptor can do the trick. There are three versions of that, which are:

  • call new once, during startup, and hang onto the object
  • call new no more than once per request, keeping the same object for the rest of the request once it’s been asked for
  • call new every time somebody asks for the object

The first one is probably most common, but it’s often nice to use the second approach so that your model can have a first class understanding of, for example, which user is currently logged in, if any, so that manages the lifecycle for you. Anything with a new method is going to work, which means any and all object-oriented stuff written according to convention in at least the past 10 years or so.

For anything else you break out Moose/Moo, and write yourself a quick normal class that wraps whatever crazy thing you’re using and now you’re back to it being easy (and you’ll probably find that class is more pleasant to use in all your code, anyway). Really, any attempt at automating the remaining cases would probably be more code to configure for whatever use-case you have than to just write the code to do it.

For example, sometimes you want a component that’s instantiated once, but then specialises itself as requested. A useful example would be “I want to keep the database connection persistent, but still have a concept of a current user to use to enforce restrictions on queries. So there’s a role called Catalyst::Component::InstancePerContext that provides that – instead of using the Adaptor’s per-request version, you use a normal adaptor, and use that role in the class it constructs and then that object will get a method called on it once per request, which can return the final model object to be used by the controller code. I’ve probably expended more characters describing it than any given implementation takes, because it’s really just the implementation of a couple of short methods and besides that, the most common case for that is DBIx::Class. There’s also a PerRequestSchema role shipped with Catalyst::model::DBIC::Schema (which is basically a DBIx::Class-specialised adaptor, remember) that reduces it to something like:

        sub _build_per_request_schema {
my ($self, $c) = @_;
$self->schema->restrict_with_object($c->user);
}

… but again the goal isn’t so much to have lots of full-featured integration code, but to minimise the need to write integration code in the first place.

In the forthcoming second part of the interview, we talk about the flexibility of Catalyst, its learning curve, Ruby on Rails, and the framework in Software Enginnering terms

 

nikosNikos Vaggalis has a BSc in Computer Science and a MSc in Interactive Multimedia. He works as a Database Developer with Linux and Ingres, and programms in both Perl and C#. He is interested in anything related to the RDBMS and loves crafting complex SQL queries for generating reports. As a journalist, he writes articles, conducts interviews and reviews technical IT books

Git Reset

2014 January 6
by Lorna Jane Mitchell

Git-ing Out Of Trouble

Git is a popular and powerful tool for managing source code, documentation, and really anything else made of text that you’d like to keep track of.  With great power comes quite a lot of complexity however, and it can be easy to get into a tangle using this tool.  With that in mind, I thought I’d share some tips for how to “undo” with git.

The closest thing to an undo command in git is git reset.  You can undo various levels of thing, right up to throwing away changes that are already in your history.  Let’s take a look at some examples, in order of severity.

 

Git Reset

Git reset without any additional arguments (technically it defaults to git reset –mixed) will simply unstage your changes.  The changes will still be there, the files won’t change, but the changes you had already staged for your next commit will no longer be staged.  Instead, you’ll see locally modified files.  This is useful if you realise that you need to commit only part of the local modifications; git reset lets you unstage everything without losing changes, and then stage the ones you want.

 

Git Reset –Hard

Using the –hard switch is more destructive.  This will discard all changes since the last commit, regardless of whether they were staged or not.  It’s relatively difficult to lose work in git, but this is an excellent way of achieving just that!  It’s very useful though when you realise you’ve gone off on a tangent or find yourself in a dead end, as git reset –hard will just put you back to where you were when you last committed.  Personally I find it helpful to commit before going for lunch, as immediately afterwards seems to be my peak time for tangents and I can then easily rescue myself.

 

Git Reset –Hard [Revision]

Using –hard with a specific SHA1 will throw away everything in your working copy including staging area, and the commits since the one you name here.  This is great if you’re regretting something, or have committed to the wrong branch (make a new branch from here, then use this technique on the existing branch to remove your accidental commits.  I do this one a lot too).  Use with caution though, if you have already pushed your branch to somewhere else, your next push will need to use the -f switch to force the push – and if anyone has pulled your changes, they are very likely to have problems so this isn’t a recommended approach for already-pushed changes.

Hopefully there are some tips there that will help you to get out of trouble in the unlikely event that you need them.  When things go wrong, stay calm and remember that it happens to the best of us!

Lorna's blog imageLorna Jane Mitchell is a web development consultant and trainer from Leeds in the UK, specialising in open source technologies, data-related problems, and APIs. She is also an open source project lead, regular conference speaker, prolific blogger, and author of PHP Web Services, published by O’Reilly.

 

Do you want to know more about Git?

Lorna will be giving a full day tutorial – ‘Git for Development Teams’ on Thursday 6th February 2014 in London. This tutorial is organized by FLOSS UK and O’Reilly UK Ltd. Click here for further details. Please note that the early bird rates are available until January 14th.