FREE hit counter and Internet traffic statistics from freestats.com

Wednesday, November 19, 2003

News from the culture war

I received an email today that said in part:

Subject: They are going to allow the "F" word on TV
Everyone,

I don't normally do this, but this is important, so if you feel strongly about this, please respond online to your local government.

Heidi

It is soooo easy to do....I did it! We need to start taking action!!!!!

********** CLICK LINK BELOW **********

http://capwiz.com/afanet/


Thinking that the report must be an urban legend I tracked it down. However, it was not a legend and the FCC did approve of Bono's use of the word at the Golden Globe Awards. In part, the FCC's Orwellian reasoning was:

"As a threshold matter, the material aired during the "Golden Globe Awards" program does not describe or depict sexual and excretory activities and organs. The word "f--" [my edit] may be crude and offensive, but, in the context presented here, did not describe sexual and excretory organs and activities. Rather, the performer used the word "f--" [my edit] as an adjective or expletive to emphasize an exclamation. Indeed, in similar circumstances, we have found that offensive language used as an insult rather than as a description of sexual or excretory activity or organs is not within the scope of the Commission's prohibition of indecent program content."

Doesn't this miss the whole point of obscene language and turn it on its head? In fact, one could argue that had the word been used to describe a biological function, it would have a claim to legitimacy since it was used in context and as slang would be most understandable to some segment of the viewing audience. Instead, the FCC says you can only use the word out its natural context as an exclamation or an insult? Am I missing something? Which is worse, having a character on "Boston Public" demean someone by calling them a penis or using the word on the Discovery Channel when describing the sexual reproduction of primates? Continuing with their tortured logic, admitting that the use of the word may be "crude and offensive" simply makes the point. Using offensive language as an exclamation or insult is the context in which obscenity occurs.

This is similar to discussions about violence on TV and in the movies. Which is more acceptable, the violence in Saving Private Ryan or in Kill Bill? In the former, the violence is in context (being historical) and serves to underscore the bravery and sacrifice of soldiers willing to follow orders and lay down their lives for a comrade. The violence in Kill Bill or the Matrix is pointless and adds nothing to the story (what there is of it anyway). That's also why I don't object to some obscenity in movies. If in context it can add to the story, if not it is at best distracting and at worse makes the movie a farce (see Pulp Fiction).

While I agree with snopes that the AFA bulletin is not strictly correct, can anyone doubt that this is the beginning of a slippery slope? The F word will undoubtedly become maintstream as other former obscenities have because of the "community standards" aspect of the FCC enforcement (in the words of the FCC the material must be "patently offensive as measured by contemporary community standards for the broadcast medium"). Without a standard with a clear definiftion, simple inertia will inevitably pull the "standard" down to the lowest common demominator (how's that for mixing metaphors). I can almost hear the writers and producers salivating over using it in the same way for shock value in an upcoming drama like "The Practice" or "West Wing".

Tuesday, November 18, 2003

Contrary Data

Here's a little push back on the Indian outsourcing sent to me from Ron Hostetter. Especially interesting was the quote:

"Indeed, 66 percent of companies surveyed were disappointed with their outsourcing contracts, said PA Consulting Group. The survey shows only 39 percent of the companies would renew contracts with their existing outsourcing suppliers, and 15 percent planned to bring services back in-house."

Monday, November 17, 2003

The Future of IT: India

Read some disturbing stuff on the outsourcing of IT jobs to India and one CEO's perspective. Hard to complete with labor costs 1/12 that in the US and a huge population where high IQs can bubble to the top.

So that raises the question: What kinds of IT jobs will there be in the US in 10 years? As with most trends the big players are getting there first and so consultancies and developers working for mid-tier and small companies will be affected later (although not in all cases for sure). True consultants rather than those who perform staff augmentation will also survive longer as well as architects and project managers who can manage the remote team. In the end, with the increase in bandwidth and innovations in collaboration technology only the enconomic development of India will stem the tide. But of course once that happens the jobs will migrate to Indonesia or some other country. Welcome to globalism.

Berroa and Neyer

I was suprised that Rob Neyer thought that Matsui should have gotten the Rookie of the Year award over the Royals Angel Berroa. His main arguments are:

  • Berroa played in a better hitter's park. In fact, over the last 2 years Kauffman has produced 30% more runs than when on the road. Not only did Berroa hit better on the road, he also hit 11 of his 17 homeruns on the road. Although Neyer dismisses the premise that some players don't take advantage of park factors it seems absurd to think (as Neyer apparently does) that Berroa would have hit very poorly at home if not for the park factor. In only 288 plate appearances I would think that whatever goes into making a park a better hitter's park (prevailing winds, good weather, short fences, fast infield, good visibility) might just be swamped by luck (especially for a player who is obviously not going to take advantage of short fences at Fenway for example). For example, assume Berroa gets lucky and gets 5 more hits at home (or has his bad luck reversed). Now his home/road split is .306/.288 and it looks like he indeed "took advantage" of the park. In the end I think a single player's half season data is too small to make such an evaluation.

  • 2 voters left Matsui off their ballots as a protest to Matsui's rookie status. While I agree that Matsui is not a rookie in the true sense he shouldn't have been left off the ballot since he qualified under the archaic rules of MLB. However, he also mentions that 2 voters left Berroa off the ballot but "at least we can guess they did so on the merits (unless they happened to be New York writers, in which case they probably left Berroa off to curry favor with the Yankee brass)." I don't buy the argument that you can excuse stupidiy more than stubborness. Voters who left Berroa off the ballot should also be stripped of their voting privileges. However, its more likely that the 2 voters in question weren't stupid but had some political/racial or other non-baseball reason for doing so.


  • Finally, Neyer doesn't address that Berroa, being a shortstop, is clearly the more valuable of the two players. After a shaky start Berroa's fielding was far above average. Berroa also played in an environment where he was more relied upon offensively, which should count for something.

    Sunday, November 16, 2003

    Rhoids Baseball

    Ok, here is the site that applies the game state information and cranks out the results. Very cool stuff.

    Pennsylvanian Fossils

    Went on a very quick (1/2 hour) fossil hunt with my kids and their cousins on Saturday. Looked at a road cut among Pennsylvanian (323-290 my b.p.) limestone and shale. Apparently, the shale represents periods when the sea water receeded and may contain river and lake deposits while the limestone represents the shallow sea of the period. Found some brachiopods and bryozoans (a net-like colony of fenestella) to be sure and perhaps a bivalve or two.

    Also found two very nice sites on the fossils and geology of the area here in northeastern Kansas when looking to determine what we found.

    Geology of Kansas
    GeoKansas

    Friday, November 14, 2003

    Antiquarian Books

    I was recently able to view some wonderful and rare antiquarian books at the rare book room of the University of Iowa Medical Library. This is the front cover and two pictures from Ulyssis Aldrovandi's 1599 "The Histroy of Monsters". Aldrovandi (1522-1605) was a first class scholar of the reniassance and collector of curiosities. He published 13 volumes based on his collections, reading, and information he picked up from others. I think this copy was from 1642 and shows mr. fancy long ears (fanesius auritus) and mr. four eyes (quatuor oculis).



    The front cover says in part:

    Ulyssis Aldrovandi
    Native (patrician) of Bologna (Bononeinsis)

    Monstorum Historia (The History of Monsters)

    with Paralipomenis's(?) History of all animals

    Bartholomeus Ambrosinus
    (and then some stuff about this guy I think like his titles, he may have been Aldorvandi's successor in Bologna who edited this volume)

    To the fair (sereniss) and unconquerable (invictum)
    Ferdinand II
    Great and manly(?) leader

    With an exceedingly large index (I love that part)

    This is the only illustration in the first edition 1628 book "An anatomical disquisition concerning the motion of the heart and blood" by William Harvey (1578-1627). In this little 68 page book Harvey was the first to publish the idea of the circulation of the blood and so this book is sometimes said to be the most important book in the history of medicine. Harvey's book is very rare and exceedingly valuable since it was printed on bad paper.

    Tony Pena

    Nice little piece on Tony Pena who won the manager of the year award in the AL. The article does fail to mention that the 16-3 start was against the White Sox, Tigers, Indians, and more of the Tigers, Indians, and White Sox. Once they went on the road against better teams they started to fail and lost 27 of their next 41. Still, a great season and a lof of fun to watch (I attended over a dozen games). From my scorebook:

    3/31/03 White Sox 2-0 W - over in 2 hours and 5 minutes, 40,302. Great performance by Hernandez, 6 IP 2 hits, McDougal gets the save and looks sharp
    4/05/03 Indians 3-1 W - 44 degrees at game time, Tucker 2b, 3b, Hernandez again 7 ip, 2 hits, McDougal Ks 2 in the ninth for the save
    4/24/03 Twins 2-1 W - Only 11,981 in attendence, Sweeney homers and the Royals are 16-3, save by McDougal again, exceptional DP turned by Febles and Berroa
    5/27/03 Mariners 6-15 L - 19 hits by the Mariners, Ichiro bats in the first 5 innings, Royals now 26-23
    6/10/03 Giants 3-7 L - Some good defense by Randa and Lopez, HR by Sweeney, Bonds walks twice, Royals now 32-31
    6/30/03 Indians 5-10 L - 2 great catches by Beltran, 3 run HR by Blake in the 8th is the backbreaker
    6/30/03 Indians 5-7 L - Royals swept now 42-38, gave up 2 HR in the first and 2 in the 6th
    7/21/03 A's 0-10 L - I'm on a losing streak here. Durazo hits 2 HR, 80 degrees and perfect weather 1:05 game
    8/26/03 Rangers 9-2 W - Royals now 1/2 game out. 3 run HR by Sweeney, 2 r HR by Palmeiro, 31st of the year. 2 great catches by Beltran, only 12,917 in attendance
    9/4/03 Diamondbacks 5-6 L - in 10 innings. Had Randy Johnson on the ropes but gave up 3 in the 7th. Bad use of pitchers by Pena. Beltran scores tying run in the 9th with a walk, steals of 2nd and 3rd and score on a sac fly. Also threw out a runner at the plate
    9/10/03 Indians 9-7 W - Royals now 74-70, 4 games out, 21,581 in attendance. Affeldt 2 scoreless innings in relief
    9/23/03 Tigers 6-15 L - Now 84-73, 5.5 games out and officially eliminated. Lima is shelled. Tigers get 3 bunt hits in a row in the 5th.

    I also attended at least 2 games I didn't score, a win over the A's in late July or early August and a win in July when Beltran saved a homerun and then hit the game winning homerun in the bottom of the 10th on a 105 degree day. Sat in the upper deck dousing myself and the kids with water.

    What's a player worth?

    Interesting article in Business Week on measuring player's performance. The buzz on the article on the SABR list and Baseballprimer.com that can be summed up as saying that this technique is nothing new being developed in 1970 and called Player Win Averages (PWA). Bennet and Albert go through the technique and add to it in Curve Ball talked about previously. Interesting that the Mets are said to be interested in hiring these guys to help them. So, my unofficial list of current teams that at least have shown an interest in sabermetrics (the Rangers did way back in the 70s) include:

    1. Red Sox - Theo Epstein and hiring Bill James
    2. A's - Bill Beane
    3. Blue Jays - JP Ricciardi
    4. Cardinals - from a post on mlb.com
    5. Mets - scuttlebut from this most recent article

    I'm drawn to these types of stats because they actually measure what really happened rather than having to assign weights to various events which is always going to be more of an approximation. Of course, stats like PWA (and RC/G for that matter) don't have that much predictive value on a per player basis. I think teams should be using these play by play stats to formulate general principles of strategy for their organization. For example, when should we sacrifice, when should we steal, when should we bring in a closer (ahead or tied and what inning?), when should we pinch hit etc. They should then hand these principles to the manager to implement and evaluate him on those. Play by play data would also be one way of quantifiably evaluating a manager's performance.

    Tuesday, November 11, 2003

    RSS Now Up and Going

    This blog now has an RSS feed provided by http://baseballblogs.org/rss/makeRSS.php. I hope since I don't write about baseball only that this is ok.

    The RSS feed is http://baseballblogs.org/rss/makeRSS.php?url=http://danagonistes.blogspot.com.

    Wilson, Consilience, and Postmodernsim

    Very interesting article Back From Chaos by Edward O. Wilson on postmodernism that chronicles the rise and fall of the Enlightenment project of bringing together all of human knowledge. Of course he views the increasing specialization of knowledge and the crumbling of a liberal education as an impediment to solving pressing issues of the day and looks for consilience (a term coined in the 1840s that means "a jumping together" and the title of one of Wilson's books) to address the problem through reductionism to a materialist core leaving only natural science and the creative arts as branches of human knowledge. In this view all other pursuits in the humanities can be reduced to brain chemistry.

    Although I don't subscribe to his reductionist views, he provides a succint definition and good arguments in attacking postmodernism (although he is willing to tolerate it as a form of dissent that sharpens rationalism). For example:

    'If these premises are correct, it follows that one culture is as good as any other in the expression of truth and morality, each in its own special way. Political multiculturalism is justified; each ethnic group and sexual preference in the community has equal validity and deserves communal support and mandated representation in educational agendas—that is, again, if the premises are correct. And they must be correct, say their promoters, because to suggest otherwise is bigotry, which is a cardinal sin. Cardinal, that is, if we agree to waive in this one instance the postmodernist prohibition against universal truth, and all agree to agree for the common good. Thus Rousseau redivivus."

    And I think he accurately locates the fall of the Enlightenment in the Jacobin Terror. However, he doesn't note how enlightnement principles survived in America largely because their basis in America was not on the "general will" of Rousseau that leads to mob rule but rather the "all men are created equal" of Jefferson that leads to accountability.

    Interestingly, when Stephen Jay Gould, evolutionist and agnostic died in May of 2002, he was working on a book that was recently published called The Hedgehog, the Fox, and the Magisters Pox. Towards the end of the book he reviews Wilson's Consilience. Gould rejects Wilson's strict reductionism which preaches that all knowledge can be reduced to a materialist core and that hence the arts, religon, poetry etc. can ultimately be grounded in brain chemistry. Gould rejects this notion on two grounds 1) Emergence - the idea that whole is not the sum of the parts and so complex constructs can't be fully explained by understanding lower-level consituents, and 2) Contingency - he views the development of these more complex systems as having a strong historical component so that even if an understanding of first principles were available, you could not explain the "how" of their development.

    The most interesting and sad part though is that he says very strongly that a total materialist philosphy cannot explain the "ought" but only the "is" or morals and ethics (sounding very much like C.S. Lewis in Mere Christianity). He calls it our "moral sense" which Lewis would call our God-given sense of the divine or the Tao. Sadly, Gould then admits that though he has struggled with this issue his entire life, he has no resolution regarding the origin of "moral sense". He therefore seems to have died a true agnostic when in fact unlike so many atheists he saw the flaw of reductionism and materialism but couldn't find his way to the alternative.

    Culture of Aesthetics?

    Interesting column by George Will that talks about the increasing importance of aesthetic concerns to our culture and economy. Hadn't thought of these apparently disparate threads in a common way before.

    Sunday, November 09, 2003

    The Rise of Theodore Roosevelt

    This is the first of the two books on TR written by Edmund Morris (also the author of Dutch), which won the Pultizer Prize when first published in 1979. This book covers Rooselvelt's birth to the day McKinley died in Buffalo in September 1901. The edition I read was revised in 2001 to conform to the style of the second book.

    Like Theodore Rex this book is very detailed on relies on personal letters, news accounts, and diaries for much of the narrative. I like the fact that there is very little interpretation and the author lets the events speak for themselves. However, you can tell the author is quite impressed with TR's almost super-human intellectual and physical energy. Several times he almost apologetically mentions some feat of TR's by assuring the readers that the best information available indicates that he indeed performed the task being described.

    Among the interesting aspects of TR Morris highlights is the dichotomy between his literary pursuits and great appetite for physical extertion. In one highly entertaining vignette Morris notes that when TR and two ranch hands tracked down and captured two thieves in the wilderness of the Dakotah territory in the early 1880s, TR occupied himself between guard duty and slogging through the wilderness for weeks by reading Tolstoy and several Greek classics he had brought along. The author's description of the blizzard of 1886-87 is also very well done.

    In short, before becoming President TR was a state assemblyman from New York, a rancher in the Dakota territory, a Civil Service commissioner under two Presdients, police commisioner of New York, Assistant Secretary of the Navy, Colonel in the militia (the Rough Riders), Governor of New York, and Vice President, and author of approximately a dozen books and scores of magazine articles. A faily active life that leaves one thinking that in the latter part of the 19th century the days must have been much longer.

    In many ways the book leads up to the charge up San Juan hill in 1898 and the description of the events rings true by balancing TR's self deprecating descriptions of his actions with quotes of those around him at the time. Morris describes TR's experience in Cuba as a catharsis of his bloodlust (also exhibited in his desire to hunt). I also liked the author's descriptions of TR's views on expansionism and "Americanism", many of which, while not politically correct by modern standards, are more nuanced than I would have expected.

    Anyway, some great insights into an interesting man. Highly recommended.

    Friday, November 07, 2003

    TechEd 2003 Talks

    Here are the TechEd 2003 presentations on MS Producer.

    Thursday, November 06, 2003

    Paper on value of SB

    http://econweb.tamu.edu/turocy/papers/runest.htm

    SAF Day - 2.2

    This afternoon's keynote was about Indigo, the communication infrastructure in the Longhorn release of Windows that was unveiled at PDC. The SDK can be found on the longhorn site. The plan is that WSE v2 and the Indigo Beta in will be released in 2004.

    The data point that caught my eye was the following: To code a demo that integrated web services with IBM WebSphere using advanced protocols the Microsoft team had to do the following:

  • In VS .NET 1.1 write 56,296 lines of code

  • In Vs .NET with WSE write 27,321 lines of code

  • In VS .NET with Indigo write 3 lines of code


  • Here's an example of a web service using Indigo:
    
    
    [Confidentiality]
    [Reliability(Gurantees.ExactlyOnce | Guarantee.InOrder)]
    [Service]
    class HelloService
    {
    }


    Note that all of the infrastructure code is abstracted in attributes and that there is a difference between a simple web service and a Service as defined in Indigo.

    All the implementation details are driven through policy configured in configuration settings. Policy can specify multiple settings (i.e. security for different clients). Policy is retrieved from a remote service in a secure fashion and then cached and referenced when a call is made.

    In Indigo behavior is WSDL, structure is XSD, and policy is WS-Policy and the services execute asynchronously but may explicitly synchronize. They all require explicit credentials for all communications.

    One of the most interesting items is that Indigo scales down to within an app using an interprocess communication channel and can also be used for peer to peer apps as well.

    Some of the Indigo terminology:
    Connector: messages, services, ports, channels

  • Messages are soap but can be encoded as binary

  • Port is a location in network space, has one or more channels

  • Messages flow through a port via channels

  • Channels may add additional processing code (i.e. reliable messaging)


  • Indigo also includes a [RemoteObject] attribute which is very similar to .NET Remoting.

    The presenter then showed an example of an app that uses Indigo that Merck is working on. This is the same app they showed at the PDC.



    SAF Day - 2

    In this morning's keynote by Dave Campbell, Dave carried on the message that Pat started yesterday called "Metropolis - Part 2".

    The major topics Dave covered can be summarized as follows:

  • Space Between Services. In this topic he covered the ideas of messages acting as the only go-between for services that are fully encapsulated. He stressed that a request/response model is the most common message model but there may be others. The response may include reference data that the request uses to make other requests. As a result, the reference data has identity and needs a versioning scheme.

  • Schema Between Services. In this topic he stressed the need for messages to be immutable once written and that each message should have a schema. The schema (typically XSD) must be able to be found in order to validate messages. Messages should have Ids as well. The schemas can themselves be extensible and so you should publish a cannonical schema and allow individual apps to build on that. However, you need to save the entire message and only process what you can understand. The design of the schema can also take an early or late bound approach. Obviously the latter allows for more flexibility at the cost of performance and complexity.

  • Service-Agents and Service-Masters. The former manages Activity related data while the latter manages Resource and Activity data. Activity data is the data required for a long running transaction, i.e. the data associated with the process of creating an order. The Resource data is the longer term business type data. Activity data should be persisted in the database because it will be needed to correlate responses in long running transactions.

  • Ownership of Data in the Enterprise. Here, Dave discussed the fact that each piece of data needs an owner and that the owner publishes changes to others who receive the update and cache it. The idea is to have a single owner and multiple copies and version the data. If changes are to be made non-owning services need to make a request to change the data which changes the version.

  • Representation of Data. This was the most interesting part where he defined 4 kinds of data: Activity, Resource, Reference, and
    Request/Response. He then talked about each type's characteristics and whether it can be normalized etc. For example, Resource data is normalized and highly concurrent whereas Reference data may not be normalized and is not updated. He then talked about how the different types of data can be represented in SQL storage and the typical formats they take (XML, objects, SQL) when used. Basically, Reference and Request/Response data should be modeled as XML whereas Activity and Resource data can be modeled as objects.

  • Tying it all together. The big three recommendations from the talk are: 1) use XML-Infosets between services, 2) Use objects to implement the business logic encapsulated within a business service, 3) Store private data and messages in SQL Server


  • At the end he covered some thoughts on business processes that covered the semantics of tentative operations, cancellation and confirmation, reorderability, and interchangeability of resources and went over a few good tips for how services should be architected to be more loosely coupled.


    After the keynote I attended two breakout sessions on Service Oriented Architecture (and a trip to the Microsoft Company Store for some $10 software) which were small groups of 15 architects and a couple Microsoft folks. Very interesting discussion that made me realize that there are more questions than answers in this space at the moment. I walked away with alot more to think about.

    One more keynote today and then back to the hotel for more work...

    SAF Day - 1

    Good keynotes from Pat Helland and Bill Gates yesterday on day 1. There are a couple hundred folks at the event and I thought Gates (and Eric Rudder) was very candid in answering questions for almost an hour. I was particularly interested in Gate's comments on the investment they've made in India for outsourced development (3 large firms). Also talked with other attendees from large ISVs that have also outsourced portions of their development to India and other places.

    Helland's talk was called "Metropolis - Part 1" where he made an analogy between the evolution of cities (and associated commerce) and the evolution of IT. Within this major analogy he made 8 other analogies to individual pieces of the physical infrastructure in the evolution of citieis to pieces within IT. These included:

  • Applications as cities. Started independent. B2b is still limited. Both are big complex evolving environments

  • Applications as factories. Like factories in the late 1800s apps are mostly independent. Useful interconnection is difficult. Tomorrow there will be rich interconnectedness. App independence is essential to get work done but interconnection has advantages in order to leverage the work of others

  • Railroads as Communcation. Movement of commodities on the railroads led to the expansion of retail. Money is in the browsing, mps, chat, images, email. Will lead to the bandwidth necessary for interconnectedness. End result is that organizations that can handle structured data will prosper

  • Manufacturing and standardization. Early 1800s = hand crafted goods, late 1800s de facto standards per industry. Standard data structures have their basis in a Web services foundation. Need horizontal and vertical standards as well. Lead to increased compatibility

  • Manufactured Assemblies and Virtual Enterprises. Requires detailed standardization and produces high value goods. Virtual enterprises rely on business function outsourcing (Logistics, HR, benefits). Leads to better quality and price. Requires standards. Allows companies to do what they do well

  • Retail and distribution as bandwidth. People coming to the stores versus bringing standardized goods to stores. Created department stores (Walmart). Mail order was driven by standardization. Today in IT we have Swivel-chair integration, Fax and pray integration, and Alt-tab integration. Tomorrow we'll have Data and operation standards, Need interchangeability, Allows pre-allocation of resources, Makes biz process efficient, Business process will grow to drive the applications as standardization grows and so Apps become commodities

  • Urban infrastructure and dealing with legacy apps. Requires retrofitting buildings. IT infrastructure issues: How do you federate identity, security, naming, directory? Retrofit will happen and web services offer hope in this regard. Who pays for this?

  • City Governance and IT Governance. Vision, planning, strategy, funding. Issues with IT governance: Who makes decisions? What are priorities? Metrics



  • Pat's assessment is that IT is currently at 1880! This is the case since Business process a gleam in our eye, Virtual enterprises getting going, and Communication and browsing well established.

    Pat then went through some practical advice for implementing services today (in the pre WS-* world). Some of his most interesting points were in regards to the need for creating a conversation ID, message ID, version ID, and sequence numbers inside your messages. In that way services can ensure once only processing of messages and processing messages in the appropriate order. Another point he made is that messages should be timestamped for validity (perhaps for infinity). The service should then log requests (before sending the response) and responses so it can replay responses if the same request comes in again. This ensures idempotence or the ability to handle a message arriving multiple times. It also improves performance by essentially caching the message in the database (storage is cheap). He also recommended not implementing distributed transactions in business services and to model each message as an atomic transaction. Reference data should also be versioned and immutable so messages can reference a particular version of the reference data. When designing a business service from existing apps you need to look for idempotent sequences of messages and define a message interface (XSD schema) for each sequence.

    At the end Pat sang a couple songs about messaging, very funny! Had a nice reception at the hotel afterwards and received a signed copy of Simon Guest's new book on J2EE interoperability with .NET.


    PDC Sessions

    Are available here. Lots of new info on Whidbey and Longhorn.

    Wednesday, November 05, 2003

    Curve Ball Info

    Here is an interview with the authors of Curve Ball and one of the author's web pages.

    Curve Ball: Baseball, Statistics, and the Role of Chance in the Game

    Just finished this excellent book by Jim Albert and Jay Bennet originally published in 2001 but updated after the 2002 season. This book starts out with a discussion of models that tabletop baseball games such as Cadaco All Star Baseball, APBA, and Strat-o-Matic use. The authors then go on to discuss statistical models and how they apply to baseball. I'll summarize some of their interesting (although not all of them definitive) conclusions:

  • Chance plays a larger role in baseball statistics than most people think. For example, using simple confidence intervals if a player has a .422 OBP in 682 plate appearances you can only conclude with 95% confidence that his true ability to get on base falls between .385 and .459. As a result, regression to the mean always needs to be considered when evaluating a player based on a single season. (chapter 3)

  • There is no general effect for the situational stats that break down a player's performance by half seasons, day and night, and grass versus turf (chapter 4)

  • Home/road differences, ground ball/fly ball differences, and hitting against the opposite handed pitcher show evidence of a general effect where the differences are +12 points, +12 points against groundball pitchers, and +15 points against the opposite hand respectively. In other words, these effects are roughly the same for everyone. Some players may have truly enhanced ability in such situations but it is not typical. (chapter 4)

  • Hitting in a specific pitch count and hitting with runners on base show some evidence for differences based on ability. Particularly the former where high homerun/strikeout hitters hit worse when down in the count. (chapter 4)
  • There is some evidence that some hitters may be streaky (chapter 5)

  • RC/G more accurately predicts team runs/game than other systems including TA (Total Average), BRA (Base Run Average) , OPS (On base + slug), DX (Scoring Index), SLG, OBP, and AVG. In fact, the accuracy of these systems is reflected in the order. Note that AVG is the least predictive but the stat that most baseball fans follow. It would be interesting to see how Extrapolated Runs would figure in this analysis. (chapter 6)

  • The authors then use regression to come up with their own system that is slightly more accurate that RC/G (Least Squares Linear Regression). Interestingly, their system is derived directly from team data from 1871 to 1999 while Runs Created is an intuitive calculation (chapter 7). They then go on to show how their model is basically the Linear Weights system developed by Pete Palmer based on ideas originally put forward by George Lindsey in 1963. Interestingly, Lindsey and Palmer's system were developed based on run production probabilities in the 24 base-out situations. Originally Lindsey used partial data from the 1959 and 1960 seasons while Palmer ran computer simulations. The regression model, however, overvalues sacrifice flys while the Linear Weights System seems to overvalue stolen bases. (chapter 7)

  • The authors go on to show that RC/G tends to be a little unrealistic (overvaluing) for players at either end of the offensive spectrum. By inserting the player in the context of an average team this bias goes away and essentially mimics the Linear Weights model. (chapter 8)

  • The authors include a great Run Potential Table from 2002











    Outs/Bases01231,21,32,31,2,3
    0.511.8961.1421.4051.5111.8381.9562.332
    1.272.5360.6820.9440.9361.1851.3581.510
    2.101.2270.3220.3630.4500.5240.6330.776


    They then include a probabality of scoring table from Lindsey's 1959 and 1960 data (I can't seem to find a more recent version of a table like this although the authors showed that Lindsey's data in general was fairly close to more recent compilation although Lindsey's sample size is not that large since there were only 16 teams during that era):











    Outs/Bases01231,21,32,31,2,3
    0.253.3960.6190.8800.6050.8700.8200.820
    1.145.2660.3900.6930.4290.6330.7300.697
    2.067.1140.2120.2620.2090.2830.3320.329


    With this data it is relatively easy to evaluate when to steal bases, when to bunt, and when to intentionally walk a batter. In short they find that stealing 2nd base is advantageous as long as the runner has a probability of success of greater than 60%. Of course, the probability threshold goes down (into to the low 50s) in the later innings when the team is simply trying to score 1 run. This is interesting since most common wisdom I've always heard says that the SB % needs to be around 67% to be effective.

    For bunting the conclusion is that only weak hitters should bunt in the early innings (per MoneyBall and most other sabermetric research) and otherwise bunting should only be done in late innings when the objective is to score a single run.

    For intentional walks they used Barry Bonds as the test case and calculated that walking Bonds only makes sense with a runner on 2nd and 2 outs or 1st and 2nd with 2 outs (using 2002 data). Other situations either definitely call for pitching to him or are too close to call statistically. (chapter 8)

  • The authors include an interesting table on the probability of the home team winning given a score differential after each inning (again from Lindsey's data). For example, given a 3-0 lead after 7 innings the home team should win 94% of the time (much to the Cubs chagrin after game 6 of the NLCS). They use this data to weight the contribution of each play a player makes. This kind of analysis will be common with the increase in play by play data. This will lead to a true picture of each player's contribution to winning games. It would be nice if STATS, Inc. or MLB would make this data freely available (retrosheet only has 1992 and prior data). (chapter 10)

  • The authors include a table constructed from their own model for the odds of teams of various abilities getting a wild card birth, winning their division, pennant, and world series. In short they note that an average team (.483 to .517 winning percentage) should be expected to get into the playoffs 19% of the time. So it is not suprising that a Wild Card team would win the World Series. (chapter 12)


  • In all, this is a great exploration of some of the core sabermetric concepts.

    Incidentally, the tables and formulas provided by the authors vindicate my earlier rant about Dusty Baker bunting in the early innings of the NLCS. The situation was Lofton on 1st nobody out, Grudzlienack batting. If Grudz attempts to bunt the probability of scoring is 36.5% calculated as p of scoring = ((ps * pss) + (pf * pfs)) where ps is the probability of a successful sacrifice, pss is the probability of scoring with a runner on 2nd and 1 out, pf is the probability of the sacrifice failing and pfs is the probability of scoring with a runner on 1st and 2 outs. So (.8 * .390) + (.2 * .266) = .365. If Grudz does not bunt the probabilty of scoring is 40.4% (including calculating a double play). This calculation is performed simply by calculating Grudz odds of walking, hitting a single etc. and then multiplying that by the probability of scoring at least one run in the base-out situation that results and then adding the products.

    Maybe not a huge difference but bunting also takes the Cubs out of the big inning moving the run potential from .896 to .682. A no-win situation. Of course in 2003 the three teams with the lowest number of sacrifices hits were the Blue Jays (11), A's (22), and Red Sox (24). All 3 of these teams have sabermetric minded GM's. Not suprisingly, the Cardinals led the NL with 87. I wonder if that will change with their new emphasis on sabermetrics per an earlier post.

    Strategic Architect Forum

    Just arrived at the SAF on the Microsoft campus. Looking forward to a stimulating three days hearing about architectural issues and new technology. First class event all the way. There was a customer event just previously here at the Microsoft Conference Center. The breakpoints I'll be attending include:

    - Realizing SOA
    - Envisioning the Service Oriented Enterprise
    - Architectural Framework
    - Identity Management
    - Software Development Lifecycle

    More reports on these later. Today's keynotes are from Pat Helland and Bill Gates.

    Monday, November 03, 2003

    Luther - 4 stars

    We saw this movie a couple week ago. Rated PG-13 for some images of dead people but certainly appropriate and educational for older kids.

    The movie stars Joseph Fiennes who was in Shakespeare in Love, Enemy at the Gates, and Elizabeth.

    It follows Luther from his conversion in 1507 during a thunderstorm through "The Great Peasant War" around 1530. They do a lot of short vignettes that cover his time at the monestary in Augsburg, his time as a student at the college in Wittenberg, when he became a professor at Wittenberg leading to the posting of the 95 theses in 1517. They then track a little more slowly through the Diet of Worms in 1521 ("here I stand, I can do no other"), his seclusion at Wittenberg writing the NT, and then fast forward again to the peasant revolts and the Augsburg confession in 1530 with Charles V.

    I thought the movie was well acted and well done (Peter Ustinov as Fredrick the Wise of Saxony was particular good and provided the humor in the movie) although it could have been a little smoother in terms of the transitions and letting the viewer know what year it was. The actors never aged so sometimes it was hard to tell how long it had been since the previous scene.

    In all, it seemed to be fairly accurate hitting all the hight points although it certainly compressed some of the events and although they tried to bring out the political context of the time, it would have been difficult to get all the intrigue in there without making it a longer and more boring movie.

    I was most impressed by the accuracy of Luther's views towards papal indulgences and the mercy of God. They did not bring out Luther's more anti-semitic views since those came at the end of his life in the 1540s. They accurately portrayed Luther as totally against the peasant revolts but did not indict him for encouraging the slaughter of the peasants (which he did after the slaughter had already begun and then later condemned the landed class for their zeal).

    All in all a good movie although it might be hard to find. We found it on one screen here in KC in our area. After the movie there were several little groups who stayed and discussed the film (all over 40 years old of course). We bumped into a couple from our church and did the same for a few minutes. Good stuff.

    Kenyans and Running

    Interesting article on Slate today Why are Kenyans Fast Runners? that summarizes the theories on why Kenyans win so many distance (800m and beyond) races. Whatever the reason there was an interesting statistical analysis done a couple years ago on this same question. The analysis assumed that Kenyans (specifically from the Nandi tribe) are genetically superior and attempted to answer questions about the frequency that Europeans or others might medal in future Olympics. The most interesting quote was this:

    "Near the limits of human performance, subtle differences between groups become greatly magnified. In world-class competition, whether for Nobel Prizes or Olympic gold, such small variations in group abilities profoundly influence tribal representation in the winner's circle."

    The analysis then goes on to show that even if the abilities of two groups differed by as much as a standard deviation, a randomly selected individual from the inferior group would still have almost a one third chance of beating the individual from the superior group. However, at the far right tail of the distribution (near the limits of human performance) the odds decrease alarmingly quickly. According to the authors of this study and others this accounts for the dominance of Kenyans in distance running, West Africans (or those of West African descent) in sprinting (all of the more than 30 men who have run the 100 meters in less than 10 seconds are of West African descent), and even in Ashkenazic Jews in winning Nobel Prizes.

    I think most Kenyans and modern westerners like the author of the Slate article would dearly like to attribute the Kenyans success primarily to hard work and cultural differences (nurture over nature). This is especially true of events like marathons that seem to require a greater work ethic and other incalcuables. However, I tend to agree with the authors of the study and conclude that a large portion of the Kenyan's success is likely genetic supported by a culture that is now proud of its success.

    So what does it all mean? Ironically, in a culture that prides itself on diversity, Americans shy away from any discussion of genetic differences between groups and label as racist anyone who questions the premise. However, at the limits (and sports seems to show this clearly, whether in team sports like basketball and football or individual sports like track) it seems to me we see those differences and would be better off to acknowledge them than hide our heads in the sand.

    Friday, October 24, 2003

    Microsoft Thoughts

    Jon Box wrote an interesting little piece on open source and the Windows OS. Also see Scoble's rant on How to Hate Microsoft and the following feedback for some enlightening views.

    My own take is that:

  • Standardization is good for everybody since it allows developers and customers to shoot for a common target in the same way that economic stability promotes liberal democracy. I'm with Jon on getting the source code for the OS. Why in the world would I want it? I'm certainly not going to change it since it takes me out of my core competency and therefore increases the cost of my products and services. Open Source is not the answer, proprietary products from organizations that implement solid and secure code based on standards is.

  • Microsoft embracing and promoting W3C standards is a logical extension of the standards they brought to the desktop by gaining market share. In the desktop world of the 1990s computers were disconnected and the shrink wrapped software model predominated. In that kind of market a single platform (the Windows API) was the only way software developers could write software that took advantage of economies of scale. In the internet world, the economies of scale are tied to internet standards (HTML, XML, SOAP etc.). Therefore Microsoft's ablity to compete in the long term is directly tied to their ability to incorporate and adhere to standards in their products (such as IE). This is why the book "Breaking Windows" by David Bank was so interesting. It shows how Microsoft almost missed the boat.

  • Most of the upgrades in products such as Office (and perhaps Longhorn?) are unncessary except those that relate directly to W3C specifications that allow other software to integrate with these products and that better enable users to work in a connected environment (like DRM). This is because the standard software products that will need to run on the desktop are now well-defined and mature (word processing, spreadsheets, games). Everything else is internet related.

  • Developer tools must move in the direction of subsuming all of the infrastructure for writing applications (navigation, caching, load balancing, personalization, security, UI widgets). I as a developer should only have to specify business rules and point the app at a datastore. Whidbey is making the first steps in this direction with the inclusion of a set of enterprise design tools and a more robust framework for ASP.NET applications.

  • Microsoft makes money by convincing users and businesses to upgrade their products. Until a different revenue model comes to the forefront, don't expect Microsoft not to continue to produce upgrades and market the hell out of them (like Office System).
  • The nature of software (being essentially free to copy and distribute) makes it unlike other industries where a monopoly of the physical infrastructure truly stifles competition (the robber barons of the 1890s). So a 95% share of the desktop doesn't necessarily make a monopoly.


  • Just my two cents.

    Wednesday, October 22, 2003

    Is this the Tiqit (ticket)?

    Heard about an interesting new mobile-style device from Tiqit. This device runs Windows XP instead of CE and includes wireless connectivity. It is sort of a Tablet PC light although I don't know (and doubt) if it runs the inking layer. The interesting thing is that the screen is simply scaled down rather than losing most of the resolution as happens with Pocket PC devices. Should be an interesting addition to the mobility space.

    How far did the Mick hit it?

    Also had an interesting email discussion recently about the distance that Mickey Mantle's famous "565" foot homerun in April of 1953 actually travelled. Along with this are other claims of prodigous Mantle blasts.

    After reading these claims I went back and looked at what Adair says in his book The Physics of Baseball about how far a ball can be hit. Based on his model he says that in order to hit an 85mph fastball 400 feet you need a bat speed of 76mph. To hit one 450 feet you need a speed of 86mph. He estimates that under normal conditions this is the limit to how far a ball can be hit. If however, the fastball is 95mph add 10 feet. If the batter pulls the ball add another 15 feet. If it's 100 degrees add another 20 feet. With a 10mph wind add another 40 feet. Hit it in Colorado and add an additional 10-15 feet. Add them all up and if everything is going your way the ball could go 550 feet.

    Since his book was published in 1990 I assume that a bat speed of 76mph must be modeled after what were perceived norms for the time. With the increase in weightlifting it is probably not uncommon to get velocities of 85mph for guys like Bonds and Sosa although I haven't seen any data on this at all (incidentally I wonder if increased swing velocities account for the increase in broken bats that announcers and players often comment on). And with every increase in velocity you multiply the force (f=mv^2).

    Sosa's homerun in Florida in July was projeted at 484 feet. Adair estimates that Mante's homerun went between 511 and 520 feet because we know Mick's blast cleared a 55 foot wall by 5 feet, 460 feet from the plate and then glanced off a beer sign. So he estimates that the ball would have landed 60 feet past the sign given an angle of descent of 45 degrees.

    Tuesday, October 21, 2003

    Compact Framework Goodies

    I'm teaching a Compact Framework course this week and was reminded that in order for developers to get all the tools Microsoft has released you'll want to get the following:

    VS .NET 2003 (of course)
    Compact Framework service pack 1
    Windows Mobile Developer Power Toys
    SDK for Pocket PC 2003 (new emulators)
    Utilities for VS .NET 2003 Add On Pak 1.1

    Code Samples
    UI Guidelines for Pocket PC
    And others found on the mobility site

    Friday, October 17, 2003

    Dusty's Failures

    I think this nicely sums up Dusty's failures.

    Thursday, October 16, 2003

    Goodbye 2003 Cubs

    Here is the best piece I've seen on the collapse of the 2003 Cubs. Very well said.

    Wednesday, October 15, 2003

    Next Year Again

    Well, the collapse is complete. I guess it was inevitable but I had a bit of hope until the last out was made. Coming into the series I said the Marlins should be the favorite since they are clearly a superior offensive team and their pitching is on a par (if you count their relievers). So the best team won I think but it doesn't help when you have leads in both game 6 and 7 and blow them with your aces on the mound. Sad day in Wrigleyville.

    Also, some poor managing by Dusty. He elected to pitch to the number 8 hitter with the pitcher on deck. That is one situation where an intentional or semi-intentional walk is called for. Being down 2 versus being down 4 makes it a completely different ballgame. He also of course should have pulled Wood much earlier, Dusty has a tendency to stick with starters too long.

    Smoking Gun

    Here it is, the interference in all its glory:
    http://www.thesmokinggun.com/archive/cubfan1.html

    Updater App Block Alternative

    A post on my recent article on the updater application block from Microsoft:

    "The .NET Application Updater block does provide a framework for a client-pull model of application deployment and upgrade, and is pretty good for some simple deployment scenarios.

    There's a company called Kinitos that offers a more comprehensive solution for the central management of .NET client applications. One of the main differences is that they enable a centrally initiated style of application management including:
    - SERVER-INITIATED install, upgrade, rollback, uninstall of client applications
    - Ability to target specific users, roles or groups with an upgrade
    - Peer-based delivery of application upgrade. (Once one client at a remote location has been upgraded, the upgrade will propagate through the local peer network rather than having all updates pulled from central server.)
    - Incremental upgrades where only changed portions of application are transmitted during upgrade
    - Central monitoring, tracking of installed version info and ability to interactively query client from central location as to its current status

    This is just one area where they make .NET smart client application development, deployment and management easier. Their platform also lets applications make online/offline/occasionally-connected operation virtually transparent to user.

    It's worth a look."


    Might be interesting....

    Tuesday, October 14, 2003

    Self Destruction

    Hard to believe the spectacle I just witnessed. In all my years of watching baseball, that was the worst game I've ever seen. The team literally self destructed. At least there's plenty of blame to go around: Prior for walking Castillo, the fan for reaching over the railing (I think the call could have gone either way but certainly the fan had to know better, Gonzales for not turning the double play, and Dusty for leaving Prior in too long and intentionally walking not one, but two batters in the inning who both ended up scoring. Intentionally walking the bases loaded is almost always a bad idea and more so when you're already behind.

    It'll be interesting to see how the Cubs respond. The bats seem to have gone silent and they've started swinging at alot of first pitches. I'm confident Wood will respond well. Can't wait for tomorrow night!

    ColdFusion to ASP.NET

    Jon Box sent me a link to a nice document comparing ColdFusion to ASP.NET. Very helpful for getting the big picture.
    Migrating from ColdFusion to ASP.NET

    Monday, October 13, 2003

    Compact Framework Book Site

    Check out the site for our new book due out this month. You can download the code and get the TOC.

    Sunday, October 12, 2003

    1984 Redux?

    Let's hope that the 4-0 win by the Marlins tonight is not the start of a repeat of 1984. I heard on NPR this week that Dennis Eckersely, who pitched for the Cubs in 84, said that on the flight out to San Diego (after the Cubs had trounced the Padres in the first two games) the team was brimming with overconfidence. He clearly thought that they were looking past the Padres and we all know what happend (Steve Garvey not withstanding). Let's hope this Cubs team has a better perspective. Of course, this team has Prior and Wood for games 6 and 7 (if needed).

    Beckett was simply too tough tonight. The Cubs just got beat by a very good pitcher who was on top of his game. Zambrano bounced back pretty well so that was encouraging. It does look as if Sammy can't hit the good fastball (94 and up) anymore. Way late most of the time as Beckett turned up the heat after the brushback pitch.

    Saturday, October 11, 2003

    Cubs up 2-1

    Great Cubs win last night 5-4 to go up 2-1 against the Marlins. I thought Dusty made the right move in leaving Remlinger in there. He looked good and was hitting his spots. Wood pitched well although I thought Leiter was correct in saying he challenged too much with 2 strikes. He was ahead in the count all night but gave up the big hits with 2 strikes.

    Both Glanville and Goodwin came through in the pinch. Goodwin especially has had several huge pinch hits in the post season and coming down the stretch. He had a very good year and is still the best bunter I've ever seen (when he played for the Royals).

    Clement goes tonight, who has been inconsistent all year. We'll see which Clement shows up tonight.

    Will and California

    George Will had an excellent column this morning regarding California's recall election. I especially like:

    "Now this exercise in "direct democracy" -- precisely what America’s Founders devised institutions to prevent -- has ended with voters, full of self pity and indignation, removing an obviously incompetent governor. They have removed him from the office to which they re-elected him after he had made his incompetence obvious by making most of the decisions that brought the voters to a boil."

    He goes on to note that Arnold is no conservative and echoes my previous post that he won't be able to get California out of its initative-created fiscal mess because the budget is largely off limits. His take is that Arnold was elected because he vowed to repeal the car tax. Rightly so but until Californians put some limits on themselves, they even the Terminator isn't going to help them.

    Friday, October 10, 2003

    Moneyball

    Cardinal fan Jon Box brought this article to my attention.

    "Without putting too fine a point on it, the Redbirds are taking their first steps into the Moneyball realm. Luhnow should bring a different perspective to a team that has not relied as heavily on statistical models and data as some other franchises in recent years."

    The Cards appear to have seen the light and will start to apply some of the sabermetric principles I mentioned in my review of Moneyball posted earlier. Good for them. It doesn't appear the Cubs are anywhere close to such an approach as their thinking appears to be very traditionalist.

    Thursday, October 09, 2003

    Photographic Memory

    I just picked up Edmund Morris' The Rise of Theodore Roosevelt. I had previously read Theodore Rex about Roosevelt's presidency and this book follows his life from birth in 1858 to when McKinley is shot in 1901. Anyway, I was fascinated by description of Roosevelt's reading habits in the prologue. A short excerpt:

    ""Reading with me is a disease." [quoting Rooselvelt] He succombs to it so totally...that nothing short of a thump on the back will regain his attention. Asked to summarize a book he has been leafing through with such apparent haste, he will do so in minute detail, often quoting the actual text. The President manages to get through at least one book a day even when he is busy. Owen Wister has lent him a book (of 300 odd pages) shortly before a full evening's entertainment at the White House, and been astonished to hear a complete review of it over breakfast...On evenings like this, when he has no official entertaining to do, Roosevelt will read two or three books entire."

    In one stretch from when he was inagurated in the fall of 1901 until about the same time the following year, Roosevelt was able to list over 114 authors that he had read, many of them multi-volume works. When asked how he could quote books that he hadn't read for 20 years, he thought for a second and said something to the effect that after thinking about the book for a second, the pages came before his mind in total.

    Gov. Arnold

    I'll have to admit that I was suprised that Arnold actually won, but then again in California anything is possible. Unfortunately, I don't think Arnold is going to be able to save California. Aside from the fact that he likely doesn't know what he's talking about, I recently read some interesting analysis of California in the book The Future of Freedom: Illiberal Democracy at home and abroad by Fareed Zakaria.

    Basically, Zakaria views California's fiscal and management troubles as largely a creation of their own making through the overuse of "direct democracy" through the ballot initiative process. Zakaria notes that since proposition 13 in 1978 which lowered taxes to 1975 levels and put a limit on the amount taxes could grow, Californians (and the nation at large) have fallen in love with bypassing legislatures and making their own laws. In the 1990s there were 378 such referendums across the country compared with 257 in the 1980s, 181 in the 1970s, and 88 in the 1960s. In the year 2000 alone there were 204!

    Zakaria feels that citizens going over the heads of the legislatures causes more problems than it solves since the normal give and take of politics is destroyed and instead politicians are mandated to act in a certain way. In the biggest example, 85 percent of California's budget is actually untouchable by Arnold and the legislature as the result of ballot initiatives (for example, proposition 98 mandates that 40% of the budget be spent on education). As a result California now spends very little on infrastructure (5% today as opposed to 22% in the 1950s). So what do they do? They propose yet another ballot iniative to allocate 1% of the budget to fix bridges etc.

    Zakaria also argues that since the normal give and take of the political process is bypassed, even iniatives that seem like a good idea to conservatives like me (ending affirmative action in prop 209 and regulating immigration in prop 187) end up having polarizing effects. He contrasts these efforts with the welfare reform package passed by congress during the Clinton administration that had the benefit of being crafted through compromise.

    I find myself agreeing with much of what Zakaria says and especially his call for a return to a "liberal democracy" with representative institutions as the founders intended. There is good reason to craft a system where cooler heads can debate and compromise. All the more so as the electorate becomes less educated and the issues become more complex (how can I as a citizen know that 40% is the right amount to spend on education, why not 37%?). This book also has just the right amount of counterintuitiveness to be on the mark. Zakaria also mentions the books Paradise Lost by Peter Shrag and Democracy Derailed by David Broder which likely illuminate this topic in more detail. If only there were time to read all the books....

    Cubs Rock Marlins

    Both games of the NLCS have been great games to watch for Cubs fans. Even though they took the loss in game 1, they battled back twice to tie the score and Sosa's homerun in the bottom of the 9th was one of the great moments in Cubs history.

    I'll have to admit that I'm one of the harshest critics of the Cubs for picking up Gonzales. He's never had an on base percentage higher than .322 in a full season and so the Cubs should have known exactly what they were going to get when they picked him up. Couple that with his large salary (4.5M) and by any reasonable measure it was a poor decision. However, in the post season anybody can get hot and right now Gonzales is leading the charge. Going to try and get a ticket for game 7 at Wrigley, not likely however.

    On another front I found some interesting statistics on the probabilities of scoring in various situations in this article. I know there are some better stats somewhere, perhaps I'll need to download some retrosheet data and run the numbers once.

    Random .NET Stuff

    Some very nice information on performance tuning of a large .NET app that Microsoft runs internally can be found here. Also ran across a nice article on generic in the CLR from the October issue of MSDN Magazine

    Looks like the gloves will be off October 27 on the next version of VS .NET and the CLR as Microsoft presents at PDC.

    Monday, October 06, 2003

    DataFactory or MSDN DAAB?

    As some of you may know last year I wrote a .NET DataFactory class for a TechEd presentation I did. The slides and code for the talk including the DataFactory in both C# and VB can be downloaded from the Atomic site. The class is also explained in chapter 18 of my book Teach Yourself ADO.NET in 21 Days.

    In any case, this class was designed to do three things:
    1) Abstract the .NET Data Provider you are using by exposing a constructor that accepts the provider name and then using reflection to instantiate the proper .NET Data Provider objects (command, connection, etc) on the fly.
    2) Allow for database independent coding by abstracting the SQL statements and data types into a set of XML files that the DataFactory calls
    3) Command object caching by keeping a set of static hashtables that reference command objects that have already been created. The DataFactory then clones the command object (using ICloneable) when needed to create a new command.

    The DataFactory is similar to the MSDN Data Access Application Block first released in April of 2002 since they both reduce the amount of code you need to write. The difference I see are:

    1) The DAAB is not .NET Data Provider independent. In their model there is a class called SqlHelper that knows how to use SqlClient. If you want other providers then you write a helper class like OdbcHelper for that (as they did in the Nile 3.0 sample app). There is no common interface they all use (which wouldn't be difficult to implement)
    2) The DAAB is not database independent. It does not abstract SQL in any way
    3) The DAAB caches parameters but not commands

    That being said, the DataFactory would be more appropriate if you know that you need to work against multiple backends. The DataFactory is more complex because of the use of XML files and I don't believe it performs quite as well as the DAAB because of using reflection.

    Is next year finally here?

    Well, the Cubs are going to the NLCS after their 5-1 victory on Sunday night. Kerry Wood and Mark Prior showed why the old adage about good pitching stopping good hitting is so true. 3 wins down 8 to go. Could this be the year?

    I thought Baker's use of the bunt in the first inning with Lofton on 2nd was an example of poor managing. By bunting in that situation you almost guarantee yourself that you'll not have a big inning. And although Miller hit a double in game 4, I wouldn't hesitate to pinch hit for him in any situation, like the one in the 9th when he struck out looking with a runner on third and less than 2 outs.

    So how to do you know that bunting in that situation is normally a bad idea? Below is a table of Net Expected Run Values calculated by Carl Morris.






    Outs/Men on Base01231,21,32,31,2,3
    0.537.9071.1381.3491.5151.7621.9572.399
    1.294.544.720.920.9681.1401.3531.617
    2.114.239.347.391.486.522.630.830

    Anyway, to answer the question posed above, Lofton was on 2nd with nobody out. That situation yields an expected run value of 1.138. By bunting the runner to third you now move the state of the game to a runner on third with one out, which yields an expected run value of .920. So you decrease your number of expected runs (the chance for a big inning), while likely increasing your chance of scoring the single run (although I would guess not by much). But what if Grudz singles and moves Lofton to third? Now the expected run value moves to 1.762. What if he strikes out (which he did)? The NERV moves down to .720. So generally speaking unless you're sure your pitcher is going to throw a shutout, I don't see any reason to bunt in that situation. Further, you'll notice that when moving from 0 outs to 1 out, you decrease the NERV by as much as a half run. Outs are the most valuable resource the offense has. Squandering them with bunts in the early innings is never a good idea. However, when you a need a run to tie the game in the late innings, then the play makes sense.

    Looking at it a different way, below is a table of linear weight run values (originally computed by Pete Palmer) that I found on the SABR list. In this matrix each situation records a context in which a batter can perform. The columns indicate their result (a walk, single, double etc.) while the values are the offensive contribution of the result (i.e. a grandslam is only woth 2.82 runs since the other runners on base deserve some credit for producing the 4 runs). Using tables likes these sabermetricians can get a feel for how valuable walks or other offensive contributions are. I believe this table was generated from data in the 1990s. This kind of complete analysis can be done using retrosheet data.











    Situation/Resultwsdthrout
    none on.30.30.49.701.00-.20
    first.36.41.881.311.77-.30
    second.20.68.991.151.64-.34
    third.19.75.881.001.57-.35
    1 and 2.58.891.551.962.37-.53
    1 and 3.38.861.331.782.19-.47
    2 and 3.221.071.401.602.01-.49
    loaded1.001.321.952.392.82-.75
    ALL.34.50.821.111.49-.30

    Friday, October 03, 2003

    Struggling with C.S. Lewis

    I just finished reading Lewis Agonistes by Lou Markos, my wife's former professor. This book was the "inspiration" for the title of this blog.

    A great book and highly recommended for anyone with a desire to understand the main ideas of Lewis and apply them in this postmodern world. In the book Lou goes step by step through the modern Christian's struggle with science, the new age, evil and suffering, the arts, and heaven and hell. By adroitly pulling key passages and ideas from the Lewis cannon he weaves an argument and a response to the challenges of both modernism and postmodernism in these key areas.

    I like the fact that the book seems to challenge the reader more and more as you progress in it (at least it did me). I once heard a lecturer say that it is lucky if a scholar has one great idea in his lifetime. Well, Lewis had several and Markos illuminates them with the ease of someone who has deep knowledge of history, theology, literature, and Lewis.

    Updater Application Block for .NET

    Here is a link to my latest article on the Updater Application Block from Microsoft. Very interesting technology that uses BITS to allow .NET applications to be auto-updating or to create an auto-updating service.

    Also see my overview of all the current application blocks. It sounds also like Microsoft will be releasing several more throughout the next year so stay tuned!

    Object-relational Mapping

    After having a discussion on object-relational mapping techniques I was reminded of Scott Ambler's work. Some very good principals here.

    Thursday, October 02, 2003

    The Law of Unintended Consequences

    I was struck by a interesting experience when walking into a book store today. As I passed the shelves of books I noticed that there were sections for History, Biography, Religon, Mystery, Romance and so on. I then noticed a section called African American. While I perused the other sections I skipped the African American section, I suppose, at least subconsciously, because being white I assumed there was nothing there germaine to me. However, after feeling a bit guilty I approached the section and skimmed the titles. Of course, the section was filled not with books only about the civil rights movement or slavery but with books by African Amercian authors in all the other genres in the rest of the store.

    So here are the questions: Why would a bookstore owner feel the need to create a section only for African American authors when authors of all other races are categorized by genre? Does creating such a section serve any purpose except allowing non-black patrons to more easily pass by some very good books and at the same time perhaps causing black patrons to miss good literature in the wider store? In answer to the first question, my guess is that the same guilt that finally caused me to skim the books in that section drove the owner of the book store to create a special African American section. And in answer to the second, the owner's actions elevated race over content (in this case of a person's writing instead of his character), which likely was exactly opposite of the intent.

    So what is the wider lesson? Like affirmative action, sometimes attempts to make concessions can end up causing a result exactly opposite to that intended (the "law of unintended consequences", see Thomas Sowell's book).

    Cretaceous Fossils

    My daughter Laura and I went fossil hunting in the Smoky Hills Chalk of western Kansas in August on our way out to Colorado. Here is the website that she and I developed for her first science project of the year (FrontPage 2000, which I was fairly impressed with after having not touched FrontPage since 1997 or so).

    The 1 day hunt was hot but fascinating. I was most struck by the abundance of large clams (Platyceramus platinus) that were eroding from almost every rock and under our feet. Some of these clams were 3 to 4 feet in diameter. I was also impressed by how difficult it is to spot some of the fossils and how a trained eye like our guide's could see bones and teeth from a distance. My main discovery of the day was a small mosasaur tooth pictured on Laura's site.

    For more information on the types of fossils you find in western Kansas see the Oceans of Kansas web site. Hope to do it again next year.

    Generics in C# and VB

    Often you'll hear someone say something like, "one of the ways that C# and VB will diverge in the future is that C# will support generics". Well, both C# and VB will support generics in the Whidbey release. The second thing you'll likely hear is someone asking, "what is a generic?"

    The best explanation I've heard (from Don Box) is that generics are simply "types with holes" that can be filled by the caller. A simple example would be a type that compares two values and returns the larger of the two (adapted from this article):


    public class Compare<ItemType, ItemType>
    {
    public ItemType Larger(ItemType data, ItemType data2)
    {
    // logic...
    }
    }

    Compare<int, int> compare = new Compare<int, int>;
    int MyInt = compare.Larger(3, 5);

    The caller can then redeclare the Compare class using floats or some other data type instead of integers. While this can be done using System.Object generics will perform better since boxing and unboxing are not required (see this article for an overview of how much this costs).

    Whidbey Rocks

    I recently attended a 3 day event at Microsoft for publishers and authors. Several of the product groups came in and gave us demos and information on the next version of Visual Studio .NET and the Windows .NET Framework.

    From all the info I have thus far Whidbey is going to be a major improvement for the lives of .NET developers everywhere. I am especially interested in the increased level of committment for the support of enterprise developers in terms of baking architectural patterns and WS-I standards into the product.

    There should be lots of good sessions on this (as well as the bits) and other topics (ASP.NET improvements are significant as well) at the PDC in October. I won't be able to attend but advanced copies of a book on the Compact Framework Jon Box and I wrote will be there. Be sure to get yours!

    Thought provoking article...

    An interesting article from First Things, October 1998
    In Defense of Disbelief

    Affirmative Action

    I watched "Law and Order" tonight while in Nashville teaching a class. The topic was affirmative action and a play on the Jayson Blair case at the New York Times. The characters on the show referenced the Michigan law school decisions handed down this summer. This summer, after the decision came down I skimmed the PDF on the supreme court web site and a piece on it.

    It sounds like they've left the door open for worse mischief. If they said that you can take race into account as a "plus factor" (the law school decision) but not use any quantitative means to determine how much of a plus factor it should be (the undergraduate decision), then you leave the door wide open for any subjective interpretation unless it starts to smell bad according to the whim of the justices (as with redistricting in the 1990s). In fact, the undergraduate decision says that diversity can "constitute a compelling state interest" while the 20 points given to minority students was "not narrowly tailored to achieve educational diversity." So what exactly does "narrowly tailored" mean?

    The strange thing is that the justices say contradictory things in the same paragraph. In the graduate school decision the opinion says that "attaining a diverse student body is at the heart of the law school's proper institutional mission" and that

    "enrolling a critical mass of minority students simply to assure some specified percentage of a particular group merely because of its ethnic origin would be patently unconstitutional. But the law school defines its critical mass concept by reference to the substantial, important, and laudable educational benefits that diversity is designed to produce including..."

    and then they go on to commit sociology by citing studies that probably have very little backing. Somehow diversity can be at the heart of a university's mission but selecting applicants to fulfill this diversity "merely" by ethnic origin is wrong? So it appears that if you set a quantitative goal and discriminate you are "patently unconstitutional" but if you hide the goal and then cite the "laudable educational benefits" you can discriminate all day long. Sounds to me like giving 20 points would be a lot more fair and at least people would know what they have to overcome.

    I think the simple reason that a plus factor type system could work in a law school is that they have fewer applicants and so can make decisions based on lots of personal information (this was specifically mandated in the Bakke case) where race would presumably play a smaller role in proportion. In an undergraduate program the sheer numbers of applicants means you need to apply some quantitative measure to see who gets in and who doesn't simply because you can't collect and process such detailed information on all applicants. I noticed that the court explicitly rejected the idea of this impracticality stating that "The fact that the implementation of a program capable of providing individualized consideration might present administrative challenges does not render constitutional an otherwise problematic system." So it sounds like if you have lots of applicants you must be officially race blind but if you don't (or have the resources to sort through it) you're free to consider race? This sounds ripe for abuse since colleges won't want to give up a race-based system and so will continue to use it as a plus but not document the fact.

    I'm of the opinion that colleges should base their admissions on test scores and extra curricular activies but not race, gender, etc. The onus should be on the high schools to produce students with equivalent test scores. Of course, if you buy the argument that there are differences in general intelligence between racial groups that at least partially get represented in SAT scores, then race-based admissions that level the scores quantitatively should be acceptable. That's a whole different story that ties into the IQ discussion...

    PreJIT on .NET

    I had a question on adding PreJIT to a Windows Installer package. See this walkthrough on MSDN for how to do this....

    Guns, Germs, and Steel

    I just had to share a short synopsis I wrote almost two years ago of one of best books I've read in the last 5 years; Guns, Germs, and Steel: The Fates of Human Societies by Jared Diamond. If you haven't read it, you should...

    This Pulitzer Prize winning book from 1997 attempts to answer the question, "why was the Spaniard Francisco Pizarro able to capture the Incan emperor Atahuallpa on November 16, 1532 within minutes of their first meeting when Pizarro commanded 168 terrified soldiers and Atahuallpa 80,000?" Of course, the broader question is why Europeans so easily conquered native peoples in many parts of the globe after 1492. Diamond's answer is that Europeans inherited guns (advanced weapons), germs (diseases that natives did not have resistance to), and steel (advanced technology including writing) as lucky accident of their environment rather than any innate difference in ability.

    He then spends most of the book discussing what he views as the four proximate causes since 13,000 BC that allowed European civilization to develop more quickly.

    1) The first is the availability of domesticable plants that allowed people in the Fertile Crescent (who spawned the eventual European societies) to move from a hunter-gatherer lifestyle to a sedentary farming lifestyle. This also allowed individuals to move away from food production and become specialists (soldiers, chiefs, etc.) which in turn drove political organization and increased population densities allowing new technologies to be invented more quickly. His contention is that while food production arose independantly in five or six areas of the world, the Fertile Crescent had far and away the best wild candidates for domestication. As a result, farming developed first in this area.

    2) The second cause is the domestication of animals that spurred both agriculture and food production. Domestic animals also infected those farming peoples with diseases to which they eventually built resistance and which proved decisive when the farmers come into contact with nomadic tribes. Once again the Fertile Crescent had many more animals that were able to be domesticated than other places. For example, the Americas contained only 1 large mammal (the llama) that could be domesticated while Africa, despite being known for large mammals, had none that were ever domesticated (and still aren't) because of various traits.

    3) The third cause is the axes of the various continents. For example, the Americas are situated in a north-south axis along with Africa while Eurasis lies east-west. A north-south axis is less desirable for the spread of populations, technology, and crops since latitudinal changes bring environmental barriers such as deserts (the Sahara), rainforests (Brazil, Congo) that are difficult to cross without adapting a different set of crops and technologies. As a result the most advanced societies of the New World (the Incan and Aztec) had no knowledge of each despite being separated by only 700 miles. Conversely, trade between mediterranean societies acted as a positive feedback mechanism between the varying cultures.

    4) The fourth cause is simply the available land mass that is able to support various cultures. Basically, the more people there are the more quickly technologies develop. In this respect Eurasia is much larger than either Africa or the Americas (with Australia very much behind).

    The author walks through each cause and provides many interesting facts and some speculation. He then devotes a chapter to each major continent and traces its history as he sees through his proximate causes and from archeology and linguistic evidence. I found the book fascinating because it provides a framework for understanding history. Its especially interesting to note where certain crops were first domesticated even though we think of them originating from different areas today (for example coffee was developed in Ethiopia, not South America). Now, when you sit down to dinner you can have a lively discussion of where all the food you're eating comes from (by the way, your spouse may find this quite annoying).

    Diamond's field work has mostly been done in New Guinea so there's an overemphasis on the migrations from China through Indonesia and New Guinea finally culminating in the population of Easter Island, but the book is still well worth reading.

    Programming or Engineering?

    I was reminded that I hadn't posted anything in my field of IT yet...so...

    Recently, on a trip to Seattle I had a chance to read Professional Software Development by Steve McConnell. McConnell's name is of course instantly recognizable through his books Code Complete, Rapid Software Development, and the Software Project Survival Guide which I've read previously.

    In this book McConnell argues for moving the state of software development from its current state as a craft or art to the more rigorous approach he terms Software Engineering (chapter 4, Softwar Engineering, Not computer Science). In this approach software development is more systematized as developers apply proven techniques and disciplines to their projects. He does this by organizing the book into four parts, The Software Tar Pit, Individual Professionalism, Organizational Professionalism, and Industry Professionalism.

    McConnell argues quite persuasivley that many of the techniques and processes ("body of knowledge") needed to deliver software on time and on budget are well known and have been for quite some time. Organizations simply don't take advantage of this body of knowledge and treat software development only as construction or coding. I was particularly struck by McConnell's analysis of the stable core of this body of knowledge and his estimate that as of 2003, 50% of the knowledge needed to develop software is in a state he calls the "stable core" (p41). In other words, the stable core is that knowledge that developers can invest in early in their careers and that doesn't change over time.

    All of this leads McConnell to the most controversial aspect of the book; his analysis that software development is progressing towards professional status (chapter 6, Novum Organum) akin to the traditional engineering, medical, and law professions. Those stable and older professions all incorporate professional education, accreditation, individual and/or organizational certification, and licensing. For my part, I'd love to see a more rigorous approach to development, and if that means licensing of some small percentage of developers (McConnell estimates around 5%) then I'm all for it.

    One of the most interesting aspects to me is his support for true software engineering degree programs rather than simply Computer Science degrees. He argues that the decrease in CS degrees issued since the mid 1980s (and only now starting to rebound), is related to their lack of relevance to the day to day work of software developers (chapter 18, Hard Knocks). I'll have to admit that I have often thought that my CS degree was ill-preparation for so many of the tasks and skills needed to develop software. I for one, would heartily like to see more software engineering degrees offered that focused not only on code creation but also estimation, design, analysis, testing, documenation, etc. Back in the day when I received my CS degree, nobody even thought there was a difference and so many students had no idea what real world software development even entailed as they set out on their first job. In fact, I distinctly recall after reading Code Complete, that this book should be required reading for every new employee in an organization since it teaches so much that a CS curriculum leaves out.

    I also was interested in his classification of organizations as "process-oriented" (NASA or Boeing) or "committment-oriented" (Microsoft). The former emphasizes repeatable processes and standard techniques while the latter looks for developers who "sign up" or commit to a project and just do what it takes to get the job done. The extremes of these two types are referred to as bureaucratic or sweatshops. The former is overly process driven, so much so that developers are bogged down in documents and meetings, while the latter requires long hours and burns people out. In my experience I see alot more organizations following the committment-oriented model for two primary reasons 1) They see how Microsoft has made it succeed, and 2) It is cheaper since they don't pay overtime and requires less initial investment by upper management. The problem however as I see it, is that as an organization grows it becomes more difficult to hire developers who can be so invested, especially as the median age of developers continue to rise (now around 34 years old with the distribution moving to the right end of the curve). And secondly, organizations rarely provide the autonomy (that equals motivation) that Microsoft does to create software. Getting an organization to move from committment to process is incredibly difficult and this is just where many organizations fail. McConnell's inclusion of his company's professional development program (chapter 16) is a breath of fresh air. Would that all organizations took this approach to developers as more than simply a set of programming skills and language profficiency!

    In all, I'd highly recommend this book anyone who looks around and sees projects out of control and wonders what could be done.

    Wednesday, October 01, 2003

    Computing Offensive Value

    One of the hallmarks of sabermetrics are the formulas sabermetricians use to estimate the offensive constribution of individual players. Bill James first introduced such a formula (Runs Created) in the late 1970s or early 1980s in his yearly Baseball Abstract, which are among the founding documents of sabermetrics. Since that time others have picked up the banner and James himself has further refined his formulas and extended them to try and calculate not only how many runs a player has produced, but how that translates into wins. In fact, last year James published Win Shares, a somewhat controversial book that catalogs these attempts.

    Well, one of the best efforts at precision in the attempts to calculate how many runs a player has produced is Extrapolated Runs (XR) developed by Jim Furtado and published in 1999.