Folksonomies - Majority Rules & Tagging Abuse

I was flipping through the Shifted Librarian's notes on Liz Lawley's keynote at this year's Internet Librarian. Lots of good stuff there, but the part that struck me was on folksonomies and tagging. First of all, let me recap from the notes:

"tagging is NOT letting you see the long tail, because good folksonomies depend on critical mass and you’ll lose the smaller pieces

nobody cares enough about that obscure government document to tag it

when building tools, can look at what people call things on del.icio.us and then use that in the tools

what happens when people start relying on these tagging tools

do I really want a majority rules approach to information retrieval?"

Now I do recognize that Tagging is not going away any time soon, and will be an option for 'rough' collection classification for the foreseeable future, but Liz Lawley's comments are simply spot on! It definitely is a 'majority rules' approach, and the gaps left in a purely user classified system could be substantial (perhaps depending on the audience?). For Librarians, our expertise revolves around 'the long tail' and our ability to locate information within the smallest niches. As such, Librarians shouldn't be too quick to buy into the use of tagging for important content collections.

For me, this is more than interesting - it's an important factor in considering Folksonomy classifications. Debatable? Perhaps, but a necessary consideration.

I haven't commented on the use of folksonomies a whole lot. While the concept isn't new to me, I do have a few concerns, and like many others, am still considering the 'when & where' of practical application.

My biggest issues surrounding Folksonomies are the lack of synonym and homonym control, and the huge potential for abuse & tag spamming. I 'might' be able to get beyond the whole lack of a 'controlled vocabulary' thing, but 'tagging abuse' is a tough one. The value of tagging (beyond offering a 'personalized classification' system - which I think is a good thing) seems to occur when the number of users hit critical mass (as noted in the quote). If we need many users to make the system work, but can't trust the users not to spam the system, there's a problem.

The long and short of it is, I can't help but be a fence sitter on this one. Folksonomies look useful: for personalized classification, for identifying preferred terms for Controlled Vocabularies (perhaps a preliminary classification stepping stone?), and for smaller 'trusted groups' of users - the potential does seem to be there...

Let's just say, I'm 'getting to yes' slowly.

Canadian Law Student Blogs - The List!

Please Note: This Canadian Law Student Blogs list has a new home on LawBlogs.ca, and is no longer being updated. Please re-direct your bookmarks.

Thanks, Steve

Curiosity has me over how many Canadian law students are out there blogging. So I emailed the owner of the best one I know - Queen's Law Life. Between the two of us, we were able to come up with the following list. Am I missing any Canadian law student blawgs? Email me.

Canadian Law Student Blogs

Active:

• Queens Law Life
• Cup of tea? Charter rights?
  (added 10/13/06)
• Canons of Destruction
  (added 07/21/06)
• The Blog of a Bay Street Articling Student
  (added 05/30/06)
• Law, Eh? Law School in Canada
  (added 01/10/06)
• Brink Review - glancing sideways at the superstructure
• Canadian Law Student Blog
• One..L..ove
• Advocatus Diaboli
• Violin Law
• Headless Drummer
• we will always be a light
• the grass is always greener
• Ali Writes...
• The Distillery
• Just another law student
• Thinking Out Loud
• Modern Life is Rubbish
• Less is more (more or less)
• sliced bread #2
• the new step (is where the wild things are)
• The Scapegoat
• Lawyerlike (added 11/25/05)

Former/Inactive/Abandoned:

• Tiger in Winter
• Where Ideas Can Hang Out and Do... Whatever!

* Updated Jan. 10/06 - List is getting bigger! .... Also, I'm not advocating any of these. Think of what you would have blogged about when you were 23! =8->

Kay, MacEwan & Riskin on KM ROI

The responses for my KM Economics post (& questions) have been amazing. I'm including below a series of responses that I found particularly helpful. Warning, long post.

First up, Stuart Kay from Baker & McKenzie, Australia offers some wise cautions to using realization rates as a KM success measure. Stuart, as many of you will recall has written a number of academic & web pieces on KM ROI, including the classic LLRX article, Cost, Value and ROI for Knowledge Management in Law Firms. I'll add my support for all of the factors Stuart mentions, and add law firm collections & bad debts to his list.

With his permission, here are Stuart's thoughts on the matter:

Dear Steve,

In theory the measurement of write downs (pre-bill value less collections) might be an indicator of the success of KM efforts. However, without interviewing the partners and lawyers who worked on each matter, how do you know if the write down was due to:

• inefficiency in production (eg lack of precedents or process - which could be improved by KM);
• a poor initial estimation of fees resulting in the necessity to write down;
• an agreement with a dissatisfied client to reduce fees;
• inappropriate delegation to an inexperienced lawyer;
• a lawyer for some perverse reason known only to themselves not using a known existing K resource (eg precedent); or
• lawyer incompetence?

The labour involved in gathering the information across all the matters in the firm in which there was a write down would be prohibitive, and there is likely to be a significant error rate due to mis-reporting of the reasons for write down (eg what lawyer is going to admit to mis-quoting, dissatisfied clients or incompetence?).

If you simply benchmark KM against gross write downs, you may find yourself ruining the credibility of your KM effort as a result of a couple of poor hires a significant proportion of whose time is written down and over which you have no control or influence. The measure will not be accurate - it will be arbitrary, and you will essentially be gambling with your credibility.

Consequently, whilst I think the write down rate theoretically could be a valid measure of the ROI for KM, the practicalities of measuring it with the necessary degree of specificity will be, in most firms, absolutely prohibitive.

I hope this helps. I'm happy to discuss it in more detail if you wish.

Stuart Kay
Knowledge Manager
Baker & McKenzie, Australia

Next up, Adam Smith Esq. himself Bruce MacEwan, who has posted on this topic previously, answered each of my questions in turn:

The point is that an effective KM program minimizes writeoffs.

To answer your questions specifically:

1) Do firms consider this 'lost time' as part of their realization rate? (My firm does, but is this common?)

They better! Or they're seriously deficient in basic financial hygiene.

2) How significant is this gap? Is this a small factor in our evaluation of KM? or an important argument?

In my experience the reality of having to write off junior associates' time means that associates cost the firm money for at least the first year or two; to the extent that period can be shortened, it will justify a large investment in KM.

3) How can this be measured to get at KM's value? Are gross measures good enough, or must it be done at a matter level?

I think the gross measures are not only "good enough," but preferable to matter by matter--different practice areas are very differently "leveraged" in terms of associate/partner hours, and obviously a highly leveraged practice will show more benefit from KM than a less-levered one.

4) Does this not get back to the 'Leverage' driver? Good training (both formal and KM resource based), and building the skills of Associates, in order to reduce Partner time in delivering legal services.

Absolutely; see above.

5) What am I missing?

Not much!

Thanks for running this by me!
Cheers and best regards,
:Bruce

Response comparison: On first glance, the two responses may look to be going in opposite directions, and in a way they are. My take is that, while measurability may pose issues, the best approaches will factor in the more extreme ends of specificity. Either go very large and use realization rates at an Industry group or Practice Area level (watching for Stuart's caveats), OR, work down to sampling (cherry picking?) the most applicable types of Matters that avoid the issues Stuart mentions.

I think one of the fundamental points missed often is that KM is not always a drag on revenue. This is just an assumption, and a wrong one in my mind. As mentioned in my original post, and now having talked with a number of KM practitioners - in the trenches, the general feeling is that KM can have a significant effect on billing efficiency. Also the fact that KM is geared towards Associates in law firms. If Partners never lever down the work, the only way to make more money is to work longer hours. I guess this all comes down to the fact that KM is not ONLY an investment for the future (as David Maister & Gerry Riskin point out), but also an investment for now.

Finally I'd like to finish things off with this, the most practical advice I received, and it came via Gerry Riskin. I asked Gerry, "Do you think that I'm missing something because I feel there's value in looking for finite measures to track these processes. It's not so much the buy-in factor or making a business case for it - my firm's been at this for 6+ years. For me, it's more about guiding the process towards success, and finding some measures that let us know if things are going in the right direction."

His response:

As for "guiding the process towards success", I believe that you are correct in looking for metrics (the objective measurements) but I believe that there is a subjective side as well.

When Dan Mahoney created the Dupont model, he dramatically reduced the cost of certain legal procedures by requiring that various law firms serving Dupont share information and precedents. There was a dramatic reduction in per case costs that he could quantify. Similarly Dupont required the use of document generation software that dramatically accelerated the rate at which newer lawyers could acquire capabilities. This very positive step was likely difficult to measure in dollars but what competitive advantage goes to those firms who accelerate the progress of lawyers through the learning curve.

So, Steven, I would say continue to look for and use metrics but keep gathering the anecdotes and archive and publish them.

Is this helpful*?

It was to me Gerry :-), and now I hope it will be for everyone else too. Thank-you to everyone who responded to my questions or sent me their thoughts.