The Risks Digest

The RISKS Digest

Forum on Risks to the Public in Computers and Related Systems

ACM Committee on Computers and Public Policy, Peter G. Neumann, moderator

Volume 10 Issue 43

Saturday 22 September 1990

Contents

o Certification
Richard Platek
Paul Tomblin
John H. Whitehouse
Alan R Kaminsky
Russell C. Sorber
John H. Whitehouse
Frank Houston
BC Tompsett
o Applicability of software curricula
Jeffrey Mogul
o Occupational Licensing (Book Review)
Tony Harminc
o Info on RISKS (comp.risks)

certification

Richard Platek <richard@hector.UUCP>
Wed, 19 Sep 90 11:17:22 EDT
Considering myself a moderate Libertarian, and hence sympathetic to arguments
based on maximizing freedom and minimizing collectivistic coercion, I would
nevertheless like to champion the cause of computer professional certification.
Certification is not meant to protect us from MIT students like Mr. Ts'o.  The
truth of the matter is that the vast majority of people building computer based
systems would never be admitted to schools anywhere as discriminating as MIT (I
am MIT Class of 61, along with John Sununu, and very proud of it).  People
aren't allowed to drive cars on the road without being "certified" yet they are
allowed to program systems whose failure could be more catastrophic than poor
driving ability.  Demanding proof of road worthiness of drivers and cars is not
an abridgment of individual liberty; it is the entry fee for participating in a
social process.  I am as Libertarian as can be but I want the driver coming at
me at 60 mph to be licensed and I want the person doing the air traffic
controller software used to land my plane to be certified.  Certification need
not be s government function. The remarks that certification would lead to a
guild do not sound negative to my ears.  Guilds guaranteed the craftsmanship of
their members.  Certification needn't restrict creativity.  It just shows one
can pass some minimum requirements criteria.  I go to homeopathic doctors who
use very non-standard forms of medicine.  Yet they all have been trained and
certified in standard medicine.  Although, I avoid standard medicine I feel
more comfortable that the doctors I do go to have satisified all the
requirements which the medical establishment has set for itself.


Re: The need for software certification

Paul Tomblin <pt@geovision.UUCP>
Tue, 18 Sep 1990 10:25:31 -0400
Theodore Ts'o <tytso@ATHENA.MIT.EDU> writes:

>I am against the "certifying" of software professionals.  My objections fall
>basically into two areas.  The first is that there is no valid way to measure
>software "competence".  How do you do it?  There are many different software
>methodolgies out there, all with their own adherents --- trying to figure out
>which ones of them are ``correct'' usually results in a religious war.
[some very valid points about different approaches deleted]

>The second general objection that I have against the certification of
>software professionals is that it might very well become a guild.  In my
>mind, there is great danger that once you have the people who are
>``IN'', they will try to maintain a competitive advantage and keep most
>other people ``OUT''.  Mr. Whitehouse has already granted that a college
>degree cannot be used to discriminate those who can program well against
>those who do not program well...

>Worst yet, it could become like many unions today, and be used to protect
>mediocrity within the group against people who are actually better qualified,
>but who aren't in the appropriate magic group...

You just have to look at professional engineering practice to see that this
doesn't need to happen.  I was a professional engineer, but I choose to make
my living in software, because I'm better at it.  Engineering
(especially Civil Engineering) is very similar to how I see the future
of software developers certification because of the following:

    1   Engineers are self regulating:  Only a _panel_ of
        engineers is fit to judge if another engineer is
        incompetent or guilty of professional malpractice.

    2   Engineers are by and large employees, rather than self
        employed like doctors or lawyers.

    3   Engineering has scope for many different approaches to the
        same problem.  A University of Waterloo grad will probably
        take a different approach to a problem than a UofToronto
        grad.  They will both come up with valid solutions to the
        problem, within the limits of human falibility.

    4   A failure of an Engineering design can be life critical,
        but as long as you followed _any_ valid design
        methodology, you will probably not be guilty of
        malpractice in the event of a failure.

Engineering is not an exclusive domain.  Anybody who passes an engineering
course, works two years in the field, and passes an ethics exam can become one.
If you don't take an engineering course, you can still become one after working
6 years and taking several exams.  My father did it that way, so I know it's
possible.  You also have to get another engineer, a co worker or supervisor to
co-sign your application.  The purpose of all this is not to restrict
membership, but just to show that you are capable of doing the work you are
being certified for.

As a Professional Engineer, I was subject to the rules of the Association of
Professional Engineers of Ontario (APEO), which has a Code of Ethics.  I was
also bound by the "Ritual of the Calling of an Engineer" (the Iron Ring).  The
"Ritual" has no legal status, but was created by Rudyard Kipling before there
was a legal status for Engineers.  Both of these were designed to stress to an
Engineer his duty, but there is an important line in the Obligation, which is
part of the "Ritual":

    For my _assured_ failures and derelictions, I ask pardon
    beforehand of my betters and my equals in my calling...

So we admit that everyone fails at some time, and we aren't going to crucify you
if you screw up, providing you did so honestly, and not because you were lazy or
unprofessional.

Disclaimer: I don't speak for the APEO, and I'm not a member any more, so things
may have changed.

Paul Tomblin, Department of Redundancy Department.
nrcaer!cognos!geovision!pt or uunet!geovision!pt


Certification

John H. Whitehouse <al357@cleveland.freenet.edu>
Wed, 19 Sep 90 08:03:09 -0400
This is a reply to Mr. Ts'o's posting in which he stated that he feared that
professional certification might lead to development of a guild mentality in
which those who are certified make the test as difficult as possible; he stated
that the free market should be left to weed out incompetency.

The ICCP takes great pains to prevent development of a guild mentality.  We
certainly wouldn't like to see something like white Mark Twain described in his
book, Life on the Mississippi; there. the riverboat pilots formed just such a
guild.  On the other hand, the ICCP takes great care in construction of its
tests.  Although the test items are written by those who currently hold
certificates, they are reviewed by a committee.  The committee verifies that
the item is correct, has only one clear answer and is not a "trick" question.
A proportion of the item pool is retired each year and some of these new items
are allowed to enter the test.  The weighting of questions on a well-publicized
outline is maintained to see that the distribution of items conforms with the
outline in the study guide.  Psychometricians evaluate each test and each item
after the fact of test administration.  They maintain careful surveillance over
reliability, validity and difficulty level.  The difficulty levels have not
changed in any consistent direction since 1962.  The pass ratio remains at
about 30 percent.

I don't understand how Mr. Ts'o's fears development of a guild mentality when
certification is and has been voluntary.

His belief that the free market should correct the problem of incompetence has
not proven itself in practice.  For the last nine years, I have been a CICS
software diagnostician.  I can say that almost every error that I have seen has
been the product of an incompetent programmer.  Some of these errors have cost
the programmers' employers upwards of $ 20,000 per incident.  No one gets
reprimanded or fired.  I recently asked a classroom full of candidate
instructors for a class on CICS Problem Determination methodology why so many
CICS programmers are flatly incompetent.  These instructors said, as if in one
voice, that the problem is due to the fact that few universities teach CICS.
The problem seems due to an inability to apply what people are taught in
school.  It is one thing to answer objective and essay questions correctly and
yet another thing to apply it in practice.  The free market fails in mid-range
and large mainframe business environments because the managers are
non-technical and we run software that people never saw in school.  The demand
for warm bodies exceeds the supply of capable people.  Because of this, the
free market cannot resolve the problem.

Mr. Ts'o's fails to see the problem.  I warned of this sort of reaction in my
posting a few days ago.  I do not believe that this problem surfaces in
academic environments and Mr. Ts'o's (at MIT, project Athena) is in just such
an environment.  I am more interested in seeing the reaction of business
systems people to the problem which I describe.  There, the managers will
seldom see the problem and the better practitioners will tend to agree with my
contentions.


Certification of Software Professionals

Kaminsky Alan R <ark@cs.rit.edu>
Wed, 19 Sep 90 11:08:00 EDT
Should there be certification of software professionals?  YES, ABSOLUTELY!

It's long past time for software development to be considered an engineering
discipline, and for software developers to consider themselves engineers.
I say this for two reasons:  (1) Like other engineering disciplines, we now
have formal and semiformal methods for carrying out all aspects of software
development--specification methods, design methods, test planning methods,
software reliability models.  Our methods are now just as mathematically
grounded as methods in other engineering disciplines.  We CAN be engineers.
(2) Like other engineering disciplines, we are engaged in constructing
artifacts that the public use and that affect the public's safety.  Other
engineers design and build roads, railroads, bridges, skyscrapers, nuclear
power plants, airplane fuselages.  We design and build nuclear power plant
controllers, airplane flight controllers, railway signaling systems, and
CAD/CAM packages that other engineers use to design their artifacts.  We
SHOULD be engineers.

But if we are software engineers in the true sense of the term, we must
expect to be treated like engineers by governments and regulatory agencies.
We must undergo certification and licensing--just as civil, electrical, and
other engineers take their Professional Engineering examination and get
certified as a Licensed Professional Engineer, or whatever the procedure is
in each state.  And we must require that all software development projects
be conducted, or at least thoroughly reviewed, by a Licensed Professional
Software Engineer, who is permitted to certify that standard (software)
engineering practices have been followed, that the artifact will perform
correctly, and that the public will be safeguarded.

Should all software practitioners undergo such certification?  NO!

Not everyone who graduates with a B.S. in engineering, and who is employed
at a company to work on engineering projects, needs to become a Licensed
Professional Engineer.  So it should be with software engineering.  You
don't think you need or want to get licensed?  Fine, don't.  There'll still
be plenty of software development work for you to do.  You'll just always
be in the position of needing a Licensed Professional Software Engineer to
certify your work (once the government wakes up and starts licensing
software engineers as they should, that is).

-Alan Kaminsky, Rochester Institute of Technology, Rochester, NY


Re: The Need for Software Certification

Russell C. Sorber <sorber@motcid.UUCP>
20 Sep 90 01:50:45 GMT
>   When will certification begin?  Probably shortly after a disaster involving
>   software that was not up to snuff and was produced in questionable fashion.

Voluntary certification of software professionals has been in existence for
several years. The Institute for Certification of Computer Professionals (ICCP,
Park Ridge, IL) receives support from the ACM, the IEEE, the DPMA, and several
other international computer professional organizations.  The literature of the
ICCP also bears the logo of the IEEE and ACM.

The certification involves an education requirement, an experience requirement,
passing a 5 hour, 5 part exam, and about $120 dollars in testing fees. I
vaguely remember a reduced fee for the unemployed but I'm not sure about that.
The exam is given at several dozen international locations twice per year.

I became certified when I noticed job listings requesting CDP's.  (Certified
Data Processors).  Some Chicago Board of Trade options traders seemed especialy
interested in certification.  This is understandable when you consider that
large fortunes are risked based partly (or solely) on the output of the
computer system.

I've also worked on projects (involving life and limb) where several key people
involved should have had more training or certification, but didn't.  I found
this to be a very scary experience.  (Scary enough so that I quit without other
work lined up)!  This experience convinced me that in certain cases,
certification should be mandatory.

Nurses, physicians, pilots, civil engineers, (even hair stylists!), are all
licensed.  Wouldn't you want the electronic instrument that monitors your
heart, or checks blood for Aids, or tells the pilot whether the landing gear is
down, to be built by licensed or certified professionals?

I know I would.

Russ Sorber, CDP                Opinions are my own.
Software Contractor currently at Motorola Inc.    sorber@marble%motcid


The need for certification

John H. Whitehouse <al357@cleveland.freenet.edu>
Thu, 20 Sep 90 08:20:38 -0400
After reading yesterday's postings, I thought it necessary to reply to a few
other concerns which I have noted running throughout the various postings
concerning this subject.

There is a concern that ICCP certification assesses examinee philosophy.  This
is generally untrue.  For the most part, the exams test definition and
recognition at a very basic level.  Some of the specialty exams go deeper, down
to ability to use a concept, but not to the level of philosophy.  It is truly
amazing that only 30 % are able to pass these exams and that only serves to
emphasize the severity of the ignorance problem.  I wish to emphasize that
philosophy is NOT tested and that the exams try very hard to avoid anything
over which there may be controversy.

Second, I note that much of the opposition argument is founded upon nothing but
fear.  This is fear of the unknown because it is clear that those who wrote
those postings were not familiar with ICCP certification.  My thanks to those
who have indicated that they have also seen this underlying thread.  I also
appreciate remarks to the effect that the opponents could make the same
argument concerning CPAs, PEs and doctors.

Two other points: ethics and continuing education.  These are two other
properties of ICCP certification.  I am aware that the professional
associations have codes of ethics, but will they kick you out for violation ?
The ICCP code of ethics is stricter than that of either ACM or DPMA and the
ICCP has revoked six or seven certificates in its history.  We are in an age of
viruses, hackers and white collar crime.  I would think that prospective
employers would view the ICCP ethics code in a most favorable light.

It has been said that the half life of knowledge in our field is three years.
The ICCP requires 120 hours of continuing education every three years.  Would
anyone of sane mind oppose continuing education ?  I would rather hire someone
who I knew to have kept current than someone whose continuing education status
was an unknown.

One added word concerning the guild mentality.  Although exam questions are
submitted by current certificate holders, they are reviewed by a committee to
assure that there is one right answer (therefore not confusing, philosophical
or controversial), then admitted to a pool of items.  Each year, about 25 % of
each exam is discarded and replaced with new items drawn from the pool.
Psychometric statistics are reviewed for any old items which are retained.
Those measures used are split-half reliability, the alpha coefficient of
reliability, a discriminant index, the Flanagan, difficulty levl and actual
counts of responses for each item.  Ther are no trick questions.  Great care is
taken to stick to the outline and to specific weightings which have been
established for outline subjects.  The difficulty level for items which have
been used before is monitored in an effort to make sure that target difficulty
levels are retained from year to year.  Those difficulty levels have carefully
been maintained at 30 % pass.


Certification of software professionals

Frank Houston <houston@itd.nrl.navy.mil>
Thu, 20 Sep 90 16:42:27 -0400
Being in the business of evaluating software systems and firms who develop
software systems, I read the commentaries on certification with great interest.
I have my own opinion, which I have discussed in this forum before and to which
I will refer presently; but first I want to add my fuel to the flames that Mr.
Ts'o ignited.

Mr. Ts'o tells us how he and a group of students got an "A" for a school
project while ignoring a great many software engineering techniques.  I
maintain that there are a great many differences between school projects and
"real world" projects.  In the "real world," software engineers and programmers
other than the originators must be able to understand, revise and maintain
programs readily and without resorting to "re-engineering" strategies.  I
wonder how Mr. Ts'o's group would have fared if in the middle of the course,
the instructor had introduced major changes to the program requirements AND
REQUIRED PAIRS OF WORKING GROUPS TO EXCHANGE PROJECT MATERIALS OR SCRAMBLED THE
GROUPS.  Or what if the instructor had given them a set of unclear requirements
and graded the groups on how well they elicited and met a set of "hidden"
requirements.  Like it or not, that is the way the software business really
works.

My point?  Software engineering is more than producing functional programs and
"error free" code although these abilities should be prized.  Error free code
is meaningless if it implements the wrong function on useless data.

I think some of Mr. Ts'o's criticism may be justified.  Version control is
indeed very important, but I would have criticized the course (as described) on
other grounds, which I prefer not to discuss.

As I have written before in this forum, I have a problem with certifying
individuals.  My concern is that certified people will be powerless without an
additional economic or regulatory lever.  I briefly described such a lever in
risks a year or so ago.  To summarize, I proposed not only individual
certification but also accreditation for firms and organizations that produce
"safety critical" software.  A firm could not be accredited for "safety
critical" systems unless it employed certified individuals and passed rigorous
and comprehensive periodic reviews.

Mr. Ts'o brings up another point.  He writes:

    >If required to, I can parrot back all of the ``right'' answers
    >on a written exam.  Those answers would also mean very little
    >about how I really go about my programming work."

Effective certification would require individuals to do more than just pass a
written test.  As I envision it, certification would involve an apprenticeship,
like the professional EIT grade or the residency for a medical specialty.  True,
the applicant would take a test; but he or she would also need certified
professionals to attest to his or her competence (and character?).  In addition,
effective certification needs rigorous renewal criteria.  Where public and
individual safety must be ensured, I think such safeguards are reasonable.  I
would not, however, suggest that such standards apply to the writers of video
games, word processors, general purpose spread sheets, and the like.

Mr. Ts'o goes on about guilds and unions and fostering mediocrity.  Well, no
system that human beings administer will be perfect.  Mediocre engineers and
doctors get licenses.  A rational system of certification will accept the
mediocre along with the excellent.  The idea is to assure some minimum level of
competence.  Occasionally some incompetents will be certified, but certificates
can be rescinded.  Engineers and doctors can lose their licenses for a variety
of reasons including incompetence.  I do not know of any system of licensure or
certification that tries to exclude top-notch people; however, most licenses
are easier to obtain if one posesses certain credentials, such as an
appropriate college degree and some relevant experience.  I would not expect
software engineering to be any different.

Frank Houston, FDA/CDRH
(These are my personal views, the customary corporate disclaimers apply.)


Software Engineer Certification (Risks 10.41)

Tompsett BC <bct@cs.hull.ac.uk>
Fri, 21 Sep 90 12:37:05 BST
As I pointed out in Risks a while ago, the UK does have a means of certifying
Software Engineers. The British Computer Society, as the Professional Society
in the UK can accredit Engineers to the qualification of Chartered Engineer
(C.Eng). This is the same C.Eng qualification that is awarded to Structural
Engineers, Aeronautical Engineers, Nuclear Engineers et. al. It is considered
the highest professional qualification an Engineer can have. There are at
present several thousand such Chartered Engineers registered through the
British Computer Society and is a large proportion of their 30,000 plus
membership.

Brian Tompsett MBCS, C.Eng, Department of Computer Science, Hull University


applicability of software curricula

Jeffrey Mogul <mogul@decwrl.dec.com>
20 Sep 1990 1841-PDT (Thursday)
To back up what zzz@NISC.SRI.COM (Michael J. Konopik) writes in RISKS 10.41:
    It would seem that Theodore was so intent on blocking out the Liskov
    philosophy of programming that he didn't hear the statement of the
    purpose of 6.170.  In fact, the same teaching strategy was applied in
    6.001 and 6.004, as well.  None of those classes taught their material
    using any "real world" languages or tools.

I took 6.170 (under a different number) the first time Prof. Liskov taught it,
in 1978.  At that time, the CLU compiler wasn't even available, so we had to
code in PL/1 (which reminds me of a RISKS-type story, but that is for another
day).  So, not only were we being encouraged to use what some people consider
an unrealistic language, but we then were able only to "pretend" that we were
using CLU.

In retrospect, this was an excellent experience for me.  Since then, I've
programmed almost exclusively in unsafe languages (assembler, C, Pascal,
Modula-2) but since I learned how to apply CLU-like discpline without being
able to rely on a compiler enforcing the rules, I think my code is much better
for it.  (I'll also note that many of the good skills I learned in that class
pertain to higher-level issues that could not be enforced by any compiler.)

This has nothing to do with whether software professionals should be certified;
but I believe my experience showed me that good skills can be taught, even
though some of my classmates never got the message.
                                                            -Jeff


Occupational Licensing ( Book Review)

Tony Harminc <ONY@vm1.mcgill.ca>
Thu, 20 Sep 90 20:59:06 EDT
The Rule of Experts - Occupational Licensing in America.  By S. David
Young.  Cato Institute, 1987.  ISBN 0-932790-62-3 (paper).  99 pages.

"Occupational regulation has served to limit consumer choice, raise
consumer costs, increase practitioner income, limit practitioner
mobility, deprive the poor of adequate service, and restrict job
opportunities for minorities -- all without a demonstrated improvement
in quality or safety."

This is the author's thesis, and he backs it up quite well in this
very readable little book.  Young concentrates on what might be called
consumer occupations - lawyers, doctors and dentists being the most
prominent.  Did you know though, that 490 occupations are licensed
in the United States while 643 require registration ?  These range from
falconers and ferret breeders to barbers and beauticians.

Conspicuously missing from discussion is engineering, which is most
often held out as an example of the 'professionalism' that programmers
should aim for.  However the chapters  'Licensing and quality',
'Licensing and information control', 'Professionals and the scope
of practice', and 'Licensing and innovation' are highly relevant
even to such a supposedly non consumer-oriented business as programming.

>From the chapter 'The Demand for Licensing':

"In the public-interest theory of licensing, regulation is introduced
for the benefit of the public at the urging of consumers or their
agents.  Government is viewed as a benevolent, if sometimes misguided,
body that seeks to maximize social welfare.  Regulations are imposed at
the urging of consumer interest groups because regulators believe,
rightly or wrongly, that efficiency or fairness or both will therefore
be enhanced."

"Critics of this hypothesis believe to the contrary, however, that
regulators' and professional groups' self-interest has been and still
is the primary motivator of regulatory legislation.  And indeed the
evidence shows that consumers rarely engage in campaigns to license
occupations.  If the purpose of licensing were to improve the quality
of service, one would expect consumers, who might be the prime beneficiaries,
to promote licensure, but licensing is systematically promoted by
practitioners ..."

The book has over eighty references -- most from the US, but several
from Canada and Europe.  A number of these attempt to make the case
*for* licensing, which Young generally demolishes quite effectively.

Recommended reading.

Tony Harminc, Ultramar Canada Inc.

Please report problems with the web pages to the maintainer

Top