Forum on Risks to the Public in Computers and Related Systems
ACM Committee on Computers and Public Policy, Peter G. Neumann, moderator
Volume 10: Issue 43
Saturday 22 September 1990
Contents
Certification- Richard Platek
Paul Tomblin
John H. Whitehouse
Alan R Kaminsky
Russell C. Sorber
John H. Whitehouse
Frank Houston
BC Tompsett - Applicability of software curricula
- Jeffrey Mogul
- Occupational Licensing (Book Review)
- Tony Harminc
- Info on RISKS (comp.risks)
certification
Richard Platek <richard@hector.UUCP>
Wed, 19 Sep 90 11:17:22 EDT
Considering myself a moderate Libertarian, and hence sympathetic to arguments based on maximizing freedom and minimizing collectivistic coercion, I would nevertheless like to champion the cause of computer professional certification. Certification is not meant to protect us from MIT students like Mr. Ts'o. The truth of the matter is that the vast majority of people building computer based systems would never be admitted to schools anywhere as discriminating as MIT (I am MIT Class of 61, along with John Sununu, and very proud of it). People aren't allowed to drive cars on the road without being "certified" yet they are allowed to program systems whose failure could be more catastrophic than poor driving ability. Demanding proof of road worthiness of drivers and cars is not an abridgment of individual liberty; it is the entry fee for participating in a social process. I am as Libertarian as can be but I want the driver coming at me at 60 mph to be licensed and I want the person doing the air traffic controller software used to land my plane to be certified. Certification need not be s government function. The remarks that certification would lead to a guild do not sound negative to my ears. Guilds guaranteed the craftsmanship of their members. Certification needn't restrict creativity. It just shows one can pass some minimum requirements criteria. I go to homeopathic doctors who use very non-standard forms of medicine. Yet they all have been trained and certified in standard medicine. Although, I avoid standard medicine I feel more comfortable that the doctors I do go to have satisified all the requirements which the medical establishment has set for itself.
Re: The need for software certification
Paul Tomblin <pt@geovision.UUCP>
Tue, 18 Sep 1990 10:25:31 -0400
Theodore Ts'o <tytso@ATHENA.MIT.EDU> writes:
>I am against the "certifying" of software professionals. My objections fall
>basically into two areas. The first is that there is no valid way to measure
>software "competence". How do you do it? There are many different software
>methodolgies out there, all with their own adherents --- trying to figure out
>which ones of them are ``correct'' usually results in a religious war.
[some very valid points about different approaches deleted]
>The second general objection that I have against the certification of
>software professionals is that it might very well become a guild. In my
>mind, there is great danger that once you have the people who are
>``IN'', they will try to maintain a competitive advantage and keep most
>other people ``OUT''. Mr. Whitehouse has already granted that a college
>degree cannot be used to discriminate those who can program well against
>those who do not program well...
>Worst yet, it could become like many unions today, and be used to protect
>mediocrity within the group against people who are actually better qualified,
>but who aren't in the appropriate magic group...
You just have to look at professional engineering practice to see that this
doesn't need to happen. I was a professional engineer, but I choose to make
my living in software, because I'm better at it. Engineering
(especially Civil Engineering) is very similar to how I see the future
of software developers certification because of the following:
1 Engineers are self regulating: Only a _panel_ of
engineers is fit to judge if another engineer is
incompetent or guilty of professional malpractice.
2 Engineers are by and large employees, rather than self
employed like doctors or lawyers.
3 Engineering has scope for many different approaches to the
same problem. A University of Waterloo grad will probably
take a different approach to a problem than a UofToronto
grad. They will both come up with valid solutions to the
problem, within the limits of human falibility.
4 A failure of an Engineering design can be life critical,
but as long as you followed _any_ valid design
methodology, you will probably not be guilty of
malpractice in the event of a failure.
Engineering is not an exclusive domain. Anybody who passes an engineering
course, works two years in the field, and passes an ethics exam can become one.
If you don't take an engineering course, you can still become one after working
6 years and taking several exams. My father did it that way, so I know it's
possible. You also have to get another engineer, a co worker or supervisor to
co-sign your application. The purpose of all this is not to restrict
membership, but just to show that you are capable of doing the work you are
being certified for.
As a Professional Engineer, I was subject to the rules of the Association of
Professional Engineers of Ontario (APEO), which has a Code of Ethics. I was
also bound by the "Ritual of the Calling of an Engineer" (the Iron Ring). The
"Ritual" has no legal status, but was created by Rudyard Kipling before there
was a legal status for Engineers. Both of these were designed to stress to an
Engineer his duty, but there is an important line in the Obligation, which is
part of the "Ritual":
For my _assured_ failures and derelictions, I ask pardon
beforehand of my betters and my equals in my calling...
So we admit that everyone fails at some time, and we aren't going to crucify you
if you screw up, providing you did so honestly, and not because you were lazy or
unprofessional.
Disclaimer: I don't speak for the APEO, and I'm not a member any more, so things
may have changed.
Paul Tomblin, Department of Redundancy Department.
nrcaer!cognos!geovision!pt or uunet!geovision!pt
Certification
John H. Whitehouse <al357@cleveland.freenet.edu>
Wed, 19 Sep 90 08:03:09 -0400
This is a reply to Mr. Ts'o's posting in which he stated that he feared that professional certification might lead to development of a guild mentality in which those who are certified make the test as difficult as possible; he stated that the free market should be left to weed out incompetency. The ICCP takes great pains to prevent development of a guild mentality. We certainly wouldn't like to see something like white Mark Twain described in his book, Life on the Mississippi; there. the riverboat pilots formed just such a guild. On the other hand, the ICCP takes great care in construction of its tests. Although the test items are written by those who currently hold certificates, they are reviewed by a committee. The committee verifies that the item is correct, has only one clear answer and is not a "trick" question. A proportion of the item pool is retired each year and some of these new items are allowed to enter the test. The weighting of questions on a well-publicized outline is maintained to see that the distribution of items conforms with the outline in the study guide. Psychometricians evaluate each test and each item after the fact of test administration. They maintain careful surveillance over reliability, validity and difficulty level. The difficulty levels have not changed in any consistent direction since 1962. The pass ratio remains at about 30 percent. I don't understand how Mr. Ts'o's fears development of a guild mentality when certification is and has been voluntary. His belief that the free market should correct the problem of incompetence has not proven itself in practice. For the last nine years, I have been a CICS software diagnostician. I can say that almost every error that I have seen has been the product of an incompetent programmer. Some of these errors have cost the programmers' employers upwards of $ 20,000 per incident. No one gets reprimanded or fired. I recently asked a classroom full of candidate instructors for a class on CICS Problem Determination methodology why so many CICS programmers are flatly incompetent. These instructors said, as if in one voice, that the problem is due to the fact that few universities teach CICS. The problem seems due to an inability to apply what people are taught in school. It is one thing to answer objective and essay questions correctly and yet another thing to apply it in practice. The free market fails in mid-range and large mainframe business environments because the managers are non-technical and we run software that people never saw in school. The demand for warm bodies exceeds the supply of capable people. Because of this, the free market cannot resolve the problem. Mr. Ts'o's fails to see the problem. I warned of this sort of reaction in my posting a few days ago. I do not believe that this problem surfaces in academic environments and Mr. Ts'o's (at MIT, project Athena) is in just such an environment. I am more interested in seeing the reaction of business systems people to the problem which I describe. There, the managers will seldom see the problem and the better practitioners will tend to agree with my contentions.
Certification of Software Professionals
Kaminsky Alan R <ark@cs.rit.edu>
Wed, 19 Sep 90 11:08:00 EDT
Should there be certification of software professionals? YES, ABSOLUTELY! It's long past time for software development to be considered an engineering discipline, and for software developers to consider themselves engineers. I say this for two reasons: (1) Like other engineering disciplines, we now have formal and semiformal methods for carrying out all aspects of software development--specification methods, design methods, test planning methods, software reliability models. Our methods are now just as mathematically grounded as methods in other engineering disciplines. We CAN be engineers. (2) Like other engineering disciplines, we are engaged in constructing artifacts that the public use and that affect the public's safety. Other engineers design and build roads, railroads, bridges, skyscrapers, nuclear power plants, airplane fuselages. We design and build nuclear power plant controllers, airplane flight controllers, railway signaling systems, and CAD/CAM packages that other engineers use to design their artifacts. We SHOULD be engineers. But if we are software engineers in the true sense of the term, we must expect to be treated like engineers by governments and regulatory agencies. We must undergo certification and licensing--just as civil, electrical, and other engineers take their Professional Engineering examination and get certified as a Licensed Professional Engineer, or whatever the procedure is in each state. And we must require that all software development projects be conducted, or at least thoroughly reviewed, by a Licensed Professional Software Engineer, who is permitted to certify that standard (software) engineering practices have been followed, that the artifact will perform correctly, and that the public will be safeguarded. Should all software practitioners undergo such certification? NO! Not everyone who graduates with a B.S. in engineering, and who is employed at a company to work on engineering projects, needs to become a Licensed Professional Engineer. So it should be with software engineering. You don't think you need or want to get licensed? Fine, don't. There'll still be plenty of software development work for you to do. You'll just always be in the position of needing a Licensed Professional Software Engineer to certify your work (once the government wakes up and starts licensing software engineers as they should, that is). -Alan Kaminsky, Rochester Institute of Technology, Rochester, NY
Re: The Need for Software Certification
Russell C. Sorber <sorber@motcid.UUCP>
20 Sep 90 01:50:45 GMT
> When will certification begin? Probably shortly after a disaster involving > software that was not up to snuff and was produced in questionable fashion. Voluntary certification of software professionals has been in existence for several years. The Institute for Certification of Computer Professionals (ICCP, Park Ridge, IL) receives support from the ACM, the IEEE, the DPMA, and several other international computer professional organizations. The literature of the ICCP also bears the logo of the IEEE and ACM. The certification involves an education requirement, an experience requirement, passing a 5 hour, 5 part exam, and about $120 dollars in testing fees. I vaguely remember a reduced fee for the unemployed but I'm not sure about that. The exam is given at several dozen international locations twice per year. I became certified when I noticed job listings requesting CDP's. (Certified Data Processors). Some Chicago Board of Trade options traders seemed especialy interested in certification. This is understandable when you consider that large fortunes are risked based partly (or solely) on the output of the computer system. I've also worked on projects (involving life and limb) where several key people involved should have had more training or certification, but didn't. I found this to be a very scary experience. (Scary enough so that I quit without other work lined up)! This experience convinced me that in certain cases, certification should be mandatory. Nurses, physicians, pilots, civil engineers, (even hair stylists!), are all licensed. Wouldn't you want the electronic instrument that monitors your heart, or checks blood for Aids, or tells the pilot whether the landing gear is down, to be built by licensed or certified professionals? I know I would. Russ Sorber, CDP Opinions are my own. Software Contractor currently at Motorola Inc. sorber@marble%motcid
The need for certification
John H. Whitehouse <al357@cleveland.freenet.edu>
Thu, 20 Sep 90 08:20:38 -0400
After reading yesterday's postings, I thought it necessary to reply to a few other concerns which I have noted running throughout the various postings concerning this subject. There is a concern that ICCP certification assesses examinee philosophy. This is generally untrue. For the most part, the exams test definition and recognition at a very basic level. Some of the specialty exams go deeper, down to ability to use a concept, but not to the level of philosophy. It is truly amazing that only 30 % are able to pass these exams and that only serves to emphasize the severity of the ignorance problem. I wish to emphasize that philosophy is NOT tested and that the exams try very hard to avoid anything over which there may be controversy. Second, I note that much of the opposition argument is founded upon nothing but fear. This is fear of the unknown because it is clear that those who wrote those postings were not familiar with ICCP certification. My thanks to those who have indicated that they have also seen this underlying thread. I also appreciate remarks to the effect that the opponents could make the same argument concerning CPAs, PEs and doctors. Two other points: ethics and continuing education. These are two other properties of ICCP certification. I am aware that the professional associations have codes of ethics, but will they kick you out for violation ? The ICCP code of ethics is stricter than that of either ACM or DPMA and the ICCP has revoked six or seven certificates in its history. We are in an age of viruses, hackers and white collar crime. I would think that prospective employers would view the ICCP ethics code in a most favorable light. It has been said that the half life of knowledge in our field is three years. The ICCP requires 120 hours of continuing education every three years. Would anyone of sane mind oppose continuing education ? I would rather hire someone who I knew to have kept current than someone whose continuing education status was an unknown. One added word concerning the guild mentality. Although exam questions are submitted by current certificate holders, they are reviewed by a committee to assure that there is one right answer (therefore not confusing, philosophical or controversial), then admitted to a pool of items. Each year, about 25 % of each exam is discarded and replaced with new items drawn from the pool. Psychometric statistics are reviewed for any old items which are retained. Those measures used are split-half reliability, the alpha coefficient of reliability, a discriminant index, the Flanagan, difficulty levl and actual counts of responses for each item. Ther are no trick questions. Great care is taken to stick to the outline and to specific weightings which have been established for outline subjects. The difficulty level for items which have been used before is monitored in an effort to make sure that target difficulty levels are retained from year to year. Those difficulty levels have carefully been maintained at 30 % pass.
Certification of software professionals
Frank Houston <houston@itd.nrl.navy.mil>
Thu, 20 Sep 90 16:42:27 -0400
Being in the business of evaluating software systems and firms who develop
software systems, I read the commentaries on certification with great interest.
I have my own opinion, which I have discussed in this forum before and to which
I will refer presently; but first I want to add my fuel to the flames that Mr.
Ts'o ignited.
Mr. Ts'o tells us how he and a group of students got an "A" for a school
project while ignoring a great many software engineering techniques. I
maintain that there are a great many differences between school projects and
"real world" projects. In the "real world," software engineers and programmers
other than the originators must be able to understand, revise and maintain
programs readily and without resorting to "re-engineering" strategies. I
wonder how Mr. Ts'o's group would have fared if in the middle of the course,
the instructor had introduced major changes to the program requirements AND
REQUIRED PAIRS OF WORKING GROUPS TO EXCHANGE PROJECT MATERIALS OR SCRAMBLED THE
GROUPS. Or what if the instructor had given them a set of unclear requirements
and graded the groups on how well they elicited and met a set of "hidden"
requirements. Like it or not, that is the way the software business really
works.
My point? Software engineering is more than producing functional programs and
"error free" code although these abilities should be prized. Error free code
is meaningless if it implements the wrong function on useless data.
I think some of Mr. Ts'o's criticism may be justified. Version control is
indeed very important, but I would have criticized the course (as described) on
other grounds, which I prefer not to discuss.
As I have written before in this forum, I have a problem with certifying
individuals. My concern is that certified people will be powerless without an
additional economic or regulatory lever. I briefly described such a lever in
risks a year or so ago. To summarize, I proposed not only individual
certification but also accreditation for firms and organizations that produce
"safety critical" software. A firm could not be accredited for "safety
critical" systems unless it employed certified individuals and passed rigorous
and comprehensive periodic reviews.
Mr. Ts'o brings up another point. He writes:
>If required to, I can parrot back all of the ``right'' answers
>on a written exam. Those answers would also mean very little
>about how I really go about my programming work."
Effective certification would require individuals to do more than just pass a
written test. As I envision it, certification would involve an apprenticeship,
like the professional EIT grade or the residency for a medical specialty. True,
the applicant would take a test; but he or she would also need certified
professionals to attest to his or her competence (and character?). In addition,
effective certification needs rigorous renewal criteria. Where public and
individual safety must be ensured, I think such safeguards are reasonable. I
would not, however, suggest that such standards apply to the writers of video
games, word processors, general purpose spread sheets, and the like.
Mr. Ts'o goes on about guilds and unions and fostering mediocrity. Well, no
system that human beings administer will be perfect. Mediocre engineers and
doctors get licenses. A rational system of certification will accept the
mediocre along with the excellent. The idea is to assure some minimum level of
competence. Occasionally some incompetents will be certified, but certificates
can be rescinded. Engineers and doctors can lose their licenses for a variety
of reasons including incompetence. I do not know of any system of licensure or
certification that tries to exclude top-notch people; however, most licenses
are easier to obtain if one posesses certain credentials, such as an
appropriate college degree and some relevant experience. I would not expect
software engineering to be any different.
Frank Houston, FDA/CDRH
(These are my personal views, the customary corporate disclaimers apply.)
Software Engineer Certification (Risks 10.41)
Tompsett BC <bct@cs.hull.ac.uk>
Fri, 21 Sep 90 12:37:05 BST
As I pointed out in Risks a while ago, the UK does have a means of certifying Software Engineers. The British Computer Society, as the Professional Society in the UK can accredit Engineers to the qualification of Chartered Engineer (C.Eng). This is the same C.Eng qualification that is awarded to Structural Engineers, Aeronautical Engineers, Nuclear Engineers et. al. It is considered the highest professional qualification an Engineer can have. There are at present several thousand such Chartered Engineers registered through the British Computer Society and is a large proportion of their 30,000 plus membership. Brian Tompsett MBCS, C.Eng, Department of Computer Science, Hull University
applicability of software curricula
Jeffrey Mogul <mogul@decwrl.dec.com>
20 Sep 1990 1841-PDT (Thursday)
To back up what zzz@NISC.SRI.COM (Michael J. Konopik) writes in RISKS 10.41: It would seem that Theodore was so intent on blocking out the Liskov philosophy of programming that he didn't hear the statement of the purpose of 6.170. In fact, the same teaching strategy was applied in 6.001 and 6.004, as well. None of those classes taught their material using any "real world" languages or tools. I took 6.170 (under a different number) the first time Prof. Liskov taught it, in 1978. At that time, the CLU compiler wasn't even available, so we had to code in PL/1 (which reminds me of a RISKS-type story, but that is for another day). So, not only were we being encouraged to use what some people consider an unrealistic language, but we then were able only to "pretend" that we were using CLU. In retrospect, this was an excellent experience for me. Since then, I've programmed almost exclusively in unsafe languages (assembler, C, Pascal, Modula-2) but since I learned how to apply CLU-like discpline without being able to rely on a compiler enforcing the rules, I think my code is much better for it. (I'll also note that many of the good skills I learned in that class pertain to higher-level issues that could not be enforced by any compiler.) This has nothing to do with whether software professionals should be certified; but I believe my experience showed me that good skills can be taught, even though some of my classmates never got the message. -Jeff
Occupational Licensing ( Book Review)
Tony Harminc <ONY@vm1.mcgill.ca>
Thu, 20 Sep 90 20:59:06 EDT
The Rule of Experts - Occupational Licensing in America. By S. David Young. Cato Institute, 1987. ISBN 0-932790-62-3 (paper). 99 pages. "Occupational regulation has served to limit consumer choice, raise consumer costs, increase practitioner income, limit practitioner mobility, deprive the poor of adequate service, and restrict job opportunities for minorities -- all without a demonstrated improvement in quality or safety." This is the author's thesis, and he backs it up quite well in this very readable little book. Young concentrates on what might be called consumer occupations - lawyers, doctors and dentists being the most prominent. Did you know though, that 490 occupations are licensed in the United States while 643 require registration ? These range from falconers and ferret breeders to barbers and beauticians. Conspicuously missing from discussion is engineering, which is most often held out as an example of the 'professionalism' that programmers should aim for. However the chapters 'Licensing and quality', 'Licensing and information control', 'Professionals and the scope of practice', and 'Licensing and innovation' are highly relevant even to such a supposedly non consumer-oriented business as programming. >From the chapter 'The Demand for Licensing': "In the public-interest theory of licensing, regulation is introduced for the benefit of the public at the urging of consumers or their agents. Government is viewed as a benevolent, if sometimes misguided, body that seeks to maximize social welfare. Regulations are imposed at the urging of consumer interest groups because regulators believe, rightly or wrongly, that efficiency or fairness or both will therefore be enhanced." "Critics of this hypothesis believe to the contrary, however, that regulators' and professional groups' self-interest has been and still is the primary motivator of regulatory legislation. And indeed the evidence shows that consumers rarely engage in campaigns to license occupations. If the purpose of licensing were to improve the quality of service, one would expect consumers, who might be the prime beneficiaries, to promote licensure, but licensing is systematically promoted by practitioners ..." The book has over eighty references -- most from the US, but several from Canada and Europe. A number of these attempt to make the case *for* licensing, which Young generally demolishes quite effectively. Recommended reading. Tony Harminc, Ultramar Canada Inc.
Report problems with the web pages to the maintainer