New thoughts and discussions: Difference between revisions

Revision as of 20:15, 11 June 2007

The Cathedral and the Bazaar

The Cathedral and the Bazaar is an essay about open source computer program projects. The original text by Eric Steven Raymond is freely available at [1].

Lessons learnt from open source projects

Every good work of software starts by scratching a developer's personal itch.
Good programmers know what to write. Great ones know what to rewrite (and reuse).
"Plan to throw one away; you will, anyhow." (Fred Brooks, The Mythical Man-Month, Chapter 11)
If you have the right attitude, interesting problems will find you.
When you lose interest in a program, your last duty to it is to hand it off to a competent successor.
Treating your users as co-developers is your least-hassle route to rapid code improvement and effective debugging.
Release early. Release often. And listen to your customers.
Given a large enough beta-tester and co-developer base, almost every problem will be characterized quickly and the fix obvious to someone. Or, less formally, "Given enough eyeballs, all bugs are shallow." I dub this: "Linus's Law".
Smart data structures and dumb code works a lot better than the other way around.
If you treat your beta-testers as if they're your most valuable resource, they will respond by becoming your most valuable resource.
The next best thing to having good ideas is recognizing good ideas from your users. Sometimes the latter is better.
Often, the most striking and innovative solutions come from realizing that your concept of the problem was wrong.
"Perfection (in design) is achieved not when there is nothing more to add, but rather when there is nothing more to take away."
Any tool should be useful in the expected way, but a truly great tool lends itself to uses you never expected.
When writing gateway software of any kind, take pains to disturb the data stream as little as possible—and never throw away information unless the recipient forces you to!
When your language is nowhere near Turing-complete, syntactic sugar can be your friend.
A security system is only as secure as its secret. Beware of pseudo-secrets.
To solve an interesting problem, start by finding a problem that is interesting to you.
Provided the development coordinator has a communications medium at least as good as the Internet, and knows how to lead without coercion, many heads are inevitably better than one.

Brooks's Law is a law describing how communication becomes more difficult when the number of people increases. It is founded on experience that bugs tend strongly to cluster at the interfaces between code written by different people, and that communications/coordination overhead on a project tends to rise with the number of interfaces between human beings. Thus, problems scale with the number of communications paths between developers, which scales as the square of the humber of developers (more precisely, according to the formula N*(N - 1)/2 where N is the number of developers).

The Brooks's Law analysis (and the resulting fear of large numbers in development groups) rests on a hidden assummption: that the communications structure of the project is necessarily a complete graph, that everybody talks to everybody else. But on open-source projects, the halo developers work on what are in effect separable parallel subtasks and interact with each other very little; code changes and bug reports stream through the core group, and only within that small core group do we pay the full Brooksian overhead.

The Linux world behaves in many respects like a free market or an ecology, a collection of selfish agents attempting to maximize utility which in the process produces a self-correcting spontaneous order more elaborate and efficient than any amount of central planning could have achieved. Here, then, is the place to seek the "principle of understanding".

The "utility function" Linux hackers are maximizing is not classically economic, but is the intangible of their own ego satisfaction and reputation among other hackers. (One may call their motivation altruistic, but this ignores the fact that altruism is itself a form of ego satisfaction for the altruist). Voluntary cultures that work this way are not actually uncommon; one other in which I have long participated is science fiction fandom, which unlike hackerdom has long explicitly recognized "egoboo" (ego-boosting, or the enhancement of one's reputation among other fans) as the basic drive behind volunteer activity.

Homesteading the Noosphere

The original text by Eric Steven Raymond is freely available at [2].

Abstract

After observing a contradiction between the official ideology defined by open-source licenses and the actual behavior of hackers, I examine the actual customs that regulate the ownership and control of open-source software. I show that they imply an underlying theory of property rights homologous to the Lockean theory of land tenure. I then relate that to an analysis of the hacker culture as a gift culture in which participants compete for prestige by giving time, energy, and creativity away. Finally, I examine the consequences of this analysis for conflict resolution in the culture, and develop some prescriptive implications.

The taboos of a culture throw its norms into sharp relief. Therefore, it will be useful later on if we summarize some important ones here:

There is strong social pressure against forking projects. It does not happen except under plea of dire necessity, with much public self-justification, and requires a renaming.
Distributing changes to a project without the cooperation of the moderators is frowned upon, except in special cases like essentially trivial porting fixes.
Removing a person's name from a project history, credits, or maintainer list is absolutely not done without the person's explicit consent.

@@ Line 1: / Line 1: @@
-This page is intended for sharing your latest Intarese related thoughts and discussion topics that are already worth disseminating among interested people, but that might not be ripe enough to deserve their own pages yet. Feel free to write and process your thoughts here, no matter how unorganized they may be. '''Please edit the page so that the latest thoughts and discussions come on the top of the page.'''
+==The Cathedral and the Bazaar==
-==New way of doing science (19.4.2007)==
+[[:en:The Cathedral and the Bazaar |The Cathedral and the Bazaar]] is an essay about open source computer program projects. The original text by [[:en:Eric_S._Raymond|Eric Steven Raymond]]
+is freely available at [http://www.catb.org/~esr/writings/cathedral-bazaar/cathedral-bazaar/ar01s05.html].
-Participants: Jouni
+Lessons learnt from open source projects
+# Every good work of software starts by scratching a developer's personal itch.
+# Good programmers know what to write. Great ones know what to rewrite (and reuse).
+# "Plan to throw one away; you will, anyhow." (Fred Brooks, The Mythical Man-Month, Chapter 11)
+# If you have the right attitude, interesting problems will find you.
+# When you lose interest in a program, your last duty to it is to hand it off to a competent successor.
+# Treating your users as co-developers is your least-hassle route to rapid code improvement and effective debugging.
+# Release early. Release often. And listen to your customers.
+# Given a large enough beta-tester and co-developer base, almost every problem will be characterized quickly and the fix obvious to someone. Or, less formally, "Given enough eyeballs, all bugs are shallow." I dub this: "Linus's Law".
+# Smart data structures and dumb code works a lot better than the other way around.
+# If you treat your beta-testers as if they're your most valuable resource, they will respond by becoming your most valuable resource.
+# The next best thing to having good ideas is recognizing good ideas from your users. Sometimes the latter is better.
+# Often, the most striking and innovative solutions come from realizing that your concept of the problem was wrong.
+# "Perfection (in design) is achieved not when there is nothing more to add, but rather when there is nothing more to take away."
+# Any tool should be useful in the expected way, but a truly great tool lends itself to uses you never expected.
+# When writing gateway software of any kind, take pains to disturb the data stream as little as possible—and never throw away information unless the recipient forces you to!
+# When your language is nowhere near Turing-complete, syntactic sugar can be your friend.
+# A security system is only as secure as its secret. Beware of pseudo-secrets.
+# To solve an interesting problem, start by finding a problem that is interesting to you.
+# Provided the development coordinator has a communications medium at least as good as the Internet, and knows how to lead without coercion, many heads are inevitably better than one.
-A new way of doing science:
-*A problem is formulated on a page in the Internet (the system should be open but require registration for edits)
-*Problem is attached with "Partners needed" sign
-*Anyone can participate and produce data and upload them
-*When finished, the result will be published in an Open Access journal (that accepts material that has already been in the Internet)
-*This should be tested with an idea that otherwise does not have resources and stays undone.
-===Possible research topic for open scientific work: European-wide composite traffic===
+Brooks's Law is a law describing how communication becomes more difficult when the number of people increases. It is founded on experience that bugs tend strongly to cluster at the interfaces between code written by different people, and that communications/coordination overhead on a project tends to rise with the number of interfaces between human beings. Thus, problems scale with the number of communications paths between developers, which scales as the square of the humber of developers (more precisely, according to the formula N*(N - 1)/2 where N is the number of developers).
-*The whole European personal (not goods) traffic should be looked at based on the composite traffic idea: trip aggregation based on information about all (most) trips.
+The Brooks's Law analysis (and the resulting fear of large numbers in development groups) rests on a hidden assummption: that the communications structure of the project is necessarily a complete graph, that everybody talks to everybody else. But on open-source projects, the halo developers work on what are in effect separable parallel subtasks and interact with each other very little; code changes and bug reports stream through the core group, and only within that small core group do we pay the full Brooksian overhead.
-*This would include air and rail traffic, ships, cars, buses, and even walking.
-*The scale of the examination should cover everything between 1 and 5000 km.
-* I would predict that there are very large synergisms waiting out there, if travel modes could be more effectively combined.
-==Argumentation about variables themselves (not their content) 18.4.2007==
+The Linux world behaves in many respects like a free market or an ecology, a collection of selfish agents attempting to maximize utility which in the process produces a self-correcting spontaneous order more elaborate and efficient than any amount of central planning could have achieved. Here, then, is the place to seek the "principle of understanding".
-The current set of rules says that "However, a new variable must not be created, if there already exists a variable with the same scope."
+The "utility function" Linux hackers are maximizing is not classically economic, but is the intangible of their own ego satisfaction and reputation among other hackers. (One may call their motivation ''altruistic'', but this ignores the fact that altruism is itself a form of ego satisfaction for the altruist). Voluntary cultures that work this way are not actually uncommon; one other in which I have long participated is science fiction fandom, which unlike hackerdom has long explicitly recognized "egoboo" (ego-boosting, or the enhancement of one's reputation among other fans) as the basic drive behind volunteer activity.
-Problems:
-* If the scope is changed by one unimportant word, it is not the same. Thus, this rule does not actually prevent formation of overlapping variables.
-*World cannot be divided into a set of exclusive and mutually exhaustive variables (variables that together cover everything but do not operlap) that would make sense in general.
-* There was no rule to deal with partially overlapping variables. (Although this need was recognised.)
-* If creation of variables is prevented, we restrict the process to things that happened to be out there first.
+==Homesteading the Noosphere ==
-Solutions:
+The original text by [[:en:Eric_S._Raymond|Eric Steven Raymond]]
-* We need rules about argumentation about the existence (not only about content) of variables. We should think what these rules are.
+is freely available at [http://catb.org/~esr/writings/homesteading/homesteading/index.html].
-* It should be possible to invalidate a variable based on some criteria and argumentation. This, not rules about creating new variables, is the way of getting rid of poor variables. The criteria could be something like these:
-** Variable is irrelevant: no causal connections to anything, and therefore it cannot be a part of any risk assessment.
-** Variable is conceptually defect: the scope does not form a coherent entity. (Not even the oldest man in the world understands what the variable is about.)
-** Variable does not have clear boundaries: it is not clear what belongs to the variable and what not.
-** Variable is not measurable: A clairvoyant cannot give an answer to the result. (Note that this does not mean that the result must be numerical! ''Excellent'' and ''red'' are valid results.)
-** Variable is inefficient. There is another way of expressing the variable, and it is more practical (e.g. less time consuming to update, or data is better available). This is always compared with another variable; at least one variable must remain valid. This may also be a case when a variable is merged to another one that has a wider scope.
-* To make this work, we need an efficient categorisation of variables. This can be an extension of the issue framing method.
-*The discussion about variables cannot be held within the variable itself (at least not always: inefficiency is always about several variables). Therefore, there should be a place for this argumentation. I suggest that this is held at the set/class/category level. This means that categories become an inseparable part of the method.
-==Properties of a good risk assessment 29.1.2007==
+'''Abstract'''
-*participants: Jouni, based on previous discussions with Mikko, Juha, and others
+After observing a contradiction between the official ideology defined by open-source licenses and the actual behavior of hackers, I examine the actual customs that regulate the ownership and control of open-source software. I show that they imply an underlying theory of property rights homologous to the Lockean theory of land tenure. I then relate that to an analysis of the hacker culture as a ''gift culture'' in which participants compete for prestige by giving time, energy, and creativity away. Finally, I examine the consequences of this analysis for conflict resolution in the culture, and develop some prescriptive implications.
-===On possible solutions===
+The taboos of a culture throw its norms into sharp relief. Therefore, it will be useful later on if we summarize some important ones here:
+* There is strong social pressure against forking projects. It does not happen except under plea of dire necessity, with much public self-justification, and requires a renaming.
-'''MOTTO: There is at least one solution to every problem, and that is the truth.'''
+* Distributing changes to a project without the cooperation of the moderators is frowned upon, except in special cases like essentially trivial porting fixes.
+* Removing a person's name from a project history, credits, or maintainer list is absolutely not done without the person's explicit consent.
-This means that because the world exists, there must be at least one coherent description of the world. It is the one that most precisely describes the true nature of the affairs. In other words, if a description is incoherent with the data, it is wrong.
-This has implications on the distribution database idea. As we are planning to build a database of all variables used in all our risk assessments, there will be an increasing number of possible conflicts between the variables. This is an optimistic idea that however complicated the truth may be, there is at least one possible way to build the distribution database. The practical problem how to find that solution may, of course, be huge. On the other hand, even if the truth is not found, it is possible to do data mining within the distribution database and locate inconsistencies. When found, these would then need more scrutiny.
-===On the generalizability of variables===
-Aim: Variables must be generalizable so that they can be used without additional knowledge of the context. In other words, the context must be described well enough inside the variable.
-&rarr; Because of this, the variables must be estimates about the truth, and not deliberate under- or overestimates. Biased estimates are common in risk assessment because usually the assessments want to avoid false negative results much more than false positive results. In other words, it is much worse if there is a risk and you don't find it than if there is no risk and you think there is.
-:&rarr; Decisions may be based on risk aversion, but the estimates of variables must be best estimates, because you cannot know which decisions will be based on the variable.
-===How to study risk assessment methods?===
-What are the properties of a good risk assessment? The answer will determine the method that should be used to reach that kind of assessments and end products.
-A problem with risk assessment is that it is more like a practical work and application of existing knowledge. Can it even be a science? How risk assessment research should be performed so that it would utilise the scientific method? A suggestion:
-*Define a research need: A problem related to risk assessment.
-*The theoretical question is: Which method solves the problem?
-*Define a research hypothesis: An answer that might solve the problem.
-*The empirical reseach on the hypothesis answers this question: Does the method work out in practice?
-:&rarr; This approach is an alternative to the demand that there should be quantitative measures for the goodness of methods. In this alternative, the methods would then be compared with each other based on their quantitative goodness. This is a problematic approach, because if the measure is not good, the answer is useless. In our suggestion we leave open the question about which method is the best. Our research is about "does this work?", and the practical question "will someone use it?" will be left to the market. We assume that there is a market of risk assessments, where decision-makers and other endusers have a demand and the risk assessors have a supply.
-Some possible research ideas, when the golden criteria for a risk assessment is that it brings useful information for decision-making.
-* Problem: The contents of a risk assessment must be acceptable to the endusers. Hypothetical solution:
-** Include enduser in the process, and collect views into the assessment.
-** Alternative: Try to anticipate the needs without consulting the endusers.
-* Problem: The measure of the risk of interest can take several forms. How to select among the possible ways to describe the risk?
-** Take the endusers' predefined view as such as the basis for the assessment. E.g. if the enduser says 10<sup>-6</sup> risk is the decision criteria, focus on estimating the exposure that causes that risk.
-** Alternative: describe the risk as such, without predefined decision criteria. This alternative would require the best (or, at least good enough) scientific estimate of the risk as a whole (e.g., in the given exposure situation).
-* Problem: Different opinions about values and scoping emerge among the endusers. How to weight the opinions?
-** If an opinion was raised, it will be given a certain weight independent on who raised the point.
-** Alternative: Opinions will be given weight based on both the content and on who raised the point.
-----
-The contents of the thoughts and discussions until 11.1. have already mainly been either outdated, developed further or merged with contents of other pages and have thus been archived. You can find the old texts [http://www.pyrkilo.fi/intarese/index.php?title=New_thoughts_and_discussions&oldid=5328 | here].

New thoughts and discussions: Difference between revisions

Revision as of 20:15, 11 June 2007

The Cathedral and the Bazaar

Homesteading the Noosphere

Navigation menu