[lpi-discuss] General comments on LPI levels

[Evan Leibovitch]

Mark Miller wrote:

>On Sat, 2005-09-17 at 16:16 -0700, ross e. brunson wrote:
>
>  
>
>>I totally disagree.  It's not a notion, it's observed behaviour in the
>>real world.  I have seen people memorize enough of the LPI questions
>>from TestKings etc. to be able pass an exam.  Those same people in a
>>practicum exam would be completely at sea and probably fake a heart
>>attack to get out of the exam.
>>    
>>
>
>Trouble is this is still anecdotal. There simply is no educational
>research to support this assumption. A person can pass the test one time
>and fail it another.  There are no psychometric measures of either the
>validity or reliability of performance testing in this domain. There are
>years of such studies with written testing. I'm not making this up, I've
>spoken to people who make their living doing exactly this. 
>  
>
In fact, the only information I had ever heard about hands-on and 
psychometrics was that hands-on exams tended to be less reliable (in the 
strict definition of "reliable" used in this field), which means that 
your mood and emotions at the time of test taking are more of a factor 
in hands-on performance than in a FITB/MC (fill-in-the-blank / 
multiple-choice) exam.

Also, anyone capable of memorizing a few hundred questions, is also 
capable of memorizing command-line sequences if the hands-on task is 
similar from one test to the next. Any belief that a hands-on exam is 
immune to braindumping is probably misguided. And one can "teach to the 
test" as easily with a hands-on as with FITB/MC.

It is clear that some people respond better to one style of exam than 
the other, but that works both ways. I don't know of any exam developer 
that offers candidates the option of both styles of exam towards the 
same credential.

The way to avoid TestKing style braindumps on FITB/MC exams is to have a 
sufficiently large item pool so that memorization is pointless.  If 
someone is capable of recalling thousands of items (I seem to remember 
that the sweet spot is at about 3,500 for LPIC1), then they probably 
deserve to pass anyway :-).

Other techniques, such as giving everyone in a room of PBT tests a 
different exam, switching around the item order and using clone items 
also throw off a lot of memorization entrants.

>>I still maintain that we should have a combination for the highest
>>level, which honestly isn't directed (with respect and kudos to our 3rd
>>world brothers) to countries with insufficient infrastructure to support
>>the b/w needed to do this online.
>>
Why is that?

The beauty of Linux is that you don't need bleeding edge hardware or 
expensive proprietary software to do some really incredible things. 
These advantages help to drastically level the playing field, and allow 
countries to create their own infrastructures without the help (or 
constraints) of outside vendors.

An area's bandwidth available to connect to some remote testing server 
says NOTHING about its peoples' ability to exploit the benefits of FOSS 
to accomplish local tasks at a very advanced level.

To say that a high-level cert should be given or denied (ie, "whether 
the program is directed at them") based on bandwidth makes sense to a 
vendor-driven cert (if they can't afford the bandwidth they can't afford 
commercial software/services). However, it need not (and IMO should not) 
affect a community program that exists to advance the craft, paying more 
attention to where Linux is used rather than where it's sold.

- Evan