Jeremy Hylton: Inconceivable View RSS

Easton School District Budget 21 Apr 2010 6:40 PM (14 years ago)

The Easton school district is in a financial mess.  The school board is considering terrible options like a tax increase of more than 10 percent or layoffs totaling more than 10 percent of the teaching staff.  This post collects a few sources of information about the budget.  (Let me know if you have more links.)
The Morning Call wrote a good summary of the last school board meeting: Ax falls on Easton schools.

The school district public finance page includes the proposed budget for 2010-2011 and the budgets from the last two years.  The auditor's report from 2009 is also posted there.  (All documents there are pdfs.)  I'm not sure how useful the auditor's report is.  There is this gem from the financial highlights on page 4:

The District's overall financial position as of June 30, 2009 continued to be very strong.

As of June 30, 2010, not so much.

The salaries of all Pennsylvania public school employees in 2009 are available.  I took a quick look at the numbers.  It seems like Easton and Nazareth have similar salaries.  Parkland clearly pays better.

School Matters (Easton) has some basic data about the schools, like enrollment and students per teacher.  Overall, the numbers seem to add up to 9,160 students.    Students per teacher ranges from 12.6 at March Elementary to 20.6 at Tracy Elementary.  Unfortunately, the data here is from 2006.  It's district financial summary says the district instructional expenses are a bit under the state average, but capital expenditures  are more than five times the state average.  (Where can we find more recent numbers?)

Python strings and bytes 31 Mar 2009 6:04 PM (16 years ago)

At the PyCon sprints, we looked into a lot of bugs in the standard library caused by interactions between strings and bytes.  (A string holds a sequence of characters.  A bytes object holds a sequence of bytes, e.g. 0-255.)  I help maintain httplib and urllib, which read raw bytes from a socket and often convert them into strings.  The details of those conversions are sometimes tricky.  The rules for strings and bytes changed drastically in Python 3.0.  Most of the standard library was converted from old to new automatically (by 2to3), and many of the times those conversions were incorrect.

A harmless example comes from httplib where an if / elif statement had tests from strings and for unicode strings.  They were both converted to test for strings by the conversion tool.  The code looked like this:

    if isinstance(buf, str):  # regular strings

In this case, the second branch could be deleted.  In other cases, the effects were harmful.  If you passed a bytes object as the body argument in an HTTP request--passing form params for a POST reply is a common case--the bytes object would be converted via str() to a string.

    >>> body = b"key=value"
    >>> str(body)
    "b'key=value'"

That is, str() uses repr() to convert bytes to a string.  That's simplfy incorrect.

It will take a long time to sort out all of these problems.  We don't have a lot of experience from application developers who are using Python 3.0, so we have to invent solutions as we go along.  We're likely to make mistakes or at least make sub-optimal API decisions.

I can of think of two things that would help us make progress. 

First, we ought to organize a systematic effort to review the standard library.  How many of the libraries have plausible tests that exercise strings and bytes?  For example, the json library was carefully tested with strings and unicode in Python 2.x.  Those have all been converted to strings, so now we have a thorough set of tests for strings and none at all for bytes.

Second, we need to collect a set of best practices for writing libraries that support bytes and unicode.  A typical pattern is that bytes get sent on the wire.  (Wires, almost by definition, send bytes.)  The applications that use the wire usually want to deal with strings, which means they need to have some way to specify an encoding to use when send to or read from the wire.  We could start by collecting all the patches and bug fixes that have gone into Python 3.1 to fix string and bytes problems with 3.0.

Coroutines in Python 30 Mar 2009 8:45 AM (16 years ago)

David Beazley gave a tutorial at coroutines at PyCon 2009.  The slides and code are available for download.  I took them home with me on the plane.  I had a fun time reading the slides and studying the code.  It's a remarkably clear explanation of how generators can be used as coroutines, starting with Python 2.5.  He runs through a good collection of examples, winding up with a simple OS-style task scheduler for cooperative multi-tasking coroutines.

One of his concluding points was really helpful for me.  I found it hard to pay a lot of attention to the evolution of PEP 342, which added coroutine support for generators.  I found it confusing that yield / generators were being extended to handle different use cases.  David clarifies it in a helpful way:

There are three main uses of yield

• Iteration (a producer of data)
• Receiving messages (a consumer)
• A trap (cooperative multitasking)

Do NOT write generator functions that try to do more than one of these at once

Pressing the Police 3 Mar 2009 4:42 AM (16 years ago)

David Simon, of Homicide, The Wire, and long ago the Baltimore Sun, wrote about the increasing secrecy of the Baltimore police in Sunday's Washington Post: In Baltimore, No One Left to Press the Police. He was making a large point about the role of newspapers.

"In an American city, a police officer with the authority to take human life can now do so in the shadows, while his higher-ups can claim that this is necessary not to avoid public accountability, but to mitigate against a nonexistent wave of threats. And the last remaining daily newspaper in town no longer has the manpower, the expertise or the institutional memory to challenge any of it."

Simon argues that there aren't any bloggers fighting to keep the city government honest.  The laws provide access to many police records, but an individual is left with little practical recourse if the police don't obey them.  (They hassle photographers taking pictures on bridges, too.)  I'm sympathetic to Simon's argument, but the primary problem is public accountability not the lack of a newspaper to provide it.

It reminded me of my own experience getting access to campus police records in college.  (Neither the crimes, nor the institutions were as threatening as they are in Baltimore.)  The M.I.T. campus police refused to show reporters its police log, a simple record of incidents and arrests.  It took us a year or more of effort to get access to them.

What was involved in getting access to those records?  The Student Press Law Center provided resources for student journalists.  I knew the basic outlines of the law, that other papers had similar problems, and that many of them prevailed in the end.  The issues were clear cut at public universities, but Massachusetts law seemed fairly clear for police at private colleges.  We asked to see the records several times--just walked into the police station and asked to see it.  We also did this a few times with the local Cambridge police, who never gave us a hard time.

I also had the help of a lawyer, a former editor-in-chief of The Tech, who helped us negotiate with the Institute.  I recall a letter he wrote to Thomas Henneberry, M.I.T.'s lawyer, requesting access to the police log: "We believe that this is an appropriate matter for injunctive relief and hope seeking such relief does not become necessary."  I delivered the letter in person to various administrators--the president, the chairman of the corporation, etc.

I certainly had instiutional support from my fellow students at the newspaper.  I think it's easier to feel confident in pressing the case when you have an organization behind you, but it's hard to quantify the effect.

Henneberry wrote a letter back to us, explaining that everything we argued was nonsense.  I don't recall exactly how we responded, but several months later the police log was opened.  We did not get any official response, but the log was available to reporters.  The police log is still published reguarly.

What's the lesson for bloggers in this?  You probably need an umbrella organization to help coordinate a campaign for open records and a lawyer willing to help you in specific cases.  You need to make a sustained effort to get access to records.  The first few times you show up, you'll simply be turned away.  It's not inconceivable that bloggers could achieve as much as the local paper in this regard.

Python 3000 3 Dec 2008 10:54 AM (16 years ago)

Python 3000 is ready! The official release may not come until tomorrow, but Barry has tagged the source and is preparing the release. We've been waiting for this release for almost nine years. The earliest reference I can find is a message from Guido to python-dev in Jan. 2000.

In the previous century, we had been thinking about Python 2 as the Python version that would break backwards compatibility to make real language improvements. Python 2 ended up being a big deal, but without too many compatibility issues. All the major changes were deferred to "Python 3000," a mythical creature. I'm eager to see what programmers make of this creature.

John Dingell 23 Nov 2008 8:49 AM (16 years ago)

I remember John Dingell from my days writing for The Tech, the campus newspaper at M.I.T. He had a reputation as a bully, who liked to hold hearings and conduct investigations on fraud and waste in science to grandstand. The two I remember best ended up being busts as far as actually uncovering waste or fraud--audits of funding by research universities and a fraud case involving a researcher in David Baltimore's lab where the all charges were eventually dismissed.

The Tech, Feb. 7, 1992

The government -- particularly Rep. John D. Dingell (D-Mich.), chairman of the Oversight andInvestigations Subcommittee -- tried to publicly embarrass MIT and other universities for alleged misuses of funds, even before formal evidence was presented before the committee.

The Tech, June 6, 1997

What had originally been a matter of Imanishi-Kari's questionable research data quickly swelled into a thorny and divisive debate over the validity of scientific research.

Baltimore derided the controversy as a witch hunt and believed that some people, like U.S. Representative John Dingell (D-Mich.), were using it unreasonably to call into question government money spent on funding research.

Election Day Recap 5 Nov 2008 6:09 AM (16 years ago)

I spent a long, busy day running the polling station for Forks Western District #1 at the Forks Community Center. It was my first presidential election as a poll worker and an interesting day. I learned a lot and continue to be dismayed about how fragile the voting process is. It's easy for individual votes to be irrevocably lost despite the best efforts of the election board. The lines and registration process make it hard for people who want to vote to actually vote, regardless of whether there votes are recorded.

Depsite all the problems, I think we had a good day. Voter turnout was about 65%, more than 1,200 people voted either in person or by absentee ballot. We had long lines, ranging from 45 minutes first thing in the morning to an hour and fifteen minutes at midday. We had no serious problems, the vast majority of people were patient and friendly.

My location was two different polling stations in the same building. This causes lots of confusion for voters, since there are two different lines and they need to be in the right line to vote. The line for our precinct was much shorter than the other line. (Some people waited almost two hours in the other line.) The two election boards worked out a deal where someone who waited in the wrong line would be taken to the front of the other line. This minimized the actual harm and kept people happy, but meant I spent a lot of the day walking back and forth between the two lines.

We had two poll watchers / observes who were form the Obama campaign. They never identified themselves as Obama supporters, but I knew one of them from the neighborhood. The other came from New York. They were incredibly helpful and were essential to keeping the lines moving smoothly. They spent the day answering questions and helping people find the right line. The staff at the community center also helped out a lot.

My biggest concern was the lost votes. The tally of voters who signed in didn't match the tally of votes recorded on the electronic voting machines. Our precinct lost 26 votes out of 1186 cast, a 2.2% loss rate. The other polling station lost 19 out of 1746 votes, a 1.1% loss rate. I didn't think to check the counters on the machine until late in the day, so I don't know what went wrong early. After we started paying extra attention, we cut the error rate to less than one percent. It was probably a combination of human factors and usability issues that caused the problems, but I can't rule out some early glitch with the machine itself. I've got more data about lost votes at the end of the post.

A Princeton report concluded that the voting machines we use were too insecure to be used in New Jersey. My experience is consistent with their complaint that: "Design flaws in the user interface of the AVC Advantage disenfranchise voters."

We had seven provisional ballots cast. In two or three of those cases, I think the people simply weren't registered correctly and the ballot with not be counted. A few others will probably work out fine. I wish there were a way for me to check.

Forks Township might need to make big changes.

• The boundaries among the four precincts are very hard to understand. We seem to have cases where neighbors vote in different precincts without being on obvious bounaries.
• The precincts are vastly different in size. West #2 has at least 50% more people than West #1.
• The precincts are too big. Several smaller precincts would mean shorter lines for voting.
• We have to produce a single numbered list of voters, which seems like a central bottleneck, and we have two books that contain the entries for voters to sign. These physical lists need to be sharded more aggressively if we the precincts stay the way they are.
• I'm not sure whether it's a good idea to have two precincts vote in one location, because people get confused about which precinct to vote in. I'd estimate we had 100 people wait in our line when they were registered in the other precinct.

My choice would be to have twice as many polling stations. There are about 10,000 people registered in Forks Township. It seems reasonable to have five to ten polling stations for them instead of just four

Here is the raw data on missing votes. I didn't think to check the machine counts until late in the day, by then 23 votes were missing. One possibility is that the machine suffered some software error early in the day that caused it to drop or not record some votes. I think that's unlikely, but I can't rule it out. I'm surprised that the operators missed so many failures. Once we started checking the counters regularly, the error rate dropped substantially (3% to less than 1%). That probably means that increased vigilance caught problems while they could still be corrected. We were also much less busy late in the day.

I recorded the total number of voters signed in and the counters on the two voting machines several times starting around 4pm. The number of missing votes goes up and down slightly, because I didn't keep careful track of people in the voting booth or waiting to get in a booth when I checked the counters. If a person was in the voting booth, they'd counted as missing according to my procedure.

4:15pm, 860 voters, 428 + 409 = 837, 23 missing
4:40pm, 903 voters, 448 + 431 = 879, 24 missing
5:08pm, 943 voters, 448 + 470 = 918, 25 missing
5:45pm, 1,012 voters, 481 + 505 = 986, 26 missing
6:05pm, 1,037 voters, 494 + 515 = 1,009, 26 missing
6:45pm, 1,097 voters, 523 + 546 = 1,069, 28 missing
7:22pm, 1,146 voters, 548 + 570 = 1,118, 28 missing
7:32pm, 1,157 voters, 553 + 575 = 1,128, 29 missing
8:30pm, 1,186 voters, 569 + 591 = 1,160, 26 missing

In the final four hours of voting, we lost three more votes, an error rate of less than 1%.

There are several possible sources of error. The least likely case is a voter who signed the book, but never actually entered the voting booth. I think this case is the least likely. Why would someone wait in line for almost an hour and then leave without trying to vote?

It's possible that the voting machine did not properly record a vote. We use Sequoia AVC Advantage voting machines. They update a counter after each recorded vote. The counter is displayed on the operator panel. The machine operator could check that the counter goes up after each vote. Perhaps the machine logs more information that might catch cases where the counter fails to behave correctly.

The most likely cause of problems is a combination of usability issues and operator errors. A user unfamiliar with the machine might leave the voting booth without pressing the right combination of buttons needed to record a vote. If the operator doesn't notice, then the intended vote is lost. We had one operator for two machines used by almost 1,200 people. It would be easy to miss a few cases where people left thinking they had voted but without actually recording a vote. Here's the process in a nusthell, along with a bunch of failure modes.

1. The operator activates the machine for a specific voter. The machine makes a chirping noise.
   
2. The voter touchs the square to the right of the candidate's name. A lighted arrow appears next to the square.
3. After at least one candidate is selected, a red vote button in the lower right of the voting booth is enabled. It lights up when it is enabled.
   
4. The voter presses the red vote button to record his or her vote. The machines makes another chirping noise.
   
5. The voting machine returns to inactive mode.

There are many failure cases:

• If the voting machine isn't activated, the touch screen still responds by turning on the lighted arrow for a second or two. The red vote button is never activated. A voter could believe that they are voting because the touch screen provides some feedback. Since the vote button never lights up, they'd never see it. In this failure mode, the voter presses a bunch of names and thinks a vote has been cast. The machine stays in the inactive state through the whole process. Since the machine ends up back in the inactive state after a successful vote, the operator can't distinguish between a succesful vote and this failure mode after the voter leaves the booth. The chirping noises should help catch this failure, but voters probably don't know to listen for it. In a busy polling station with several machines, it's easy to get confused about whether you heard something or didn't.
  
• A voter can entered an activated machine and fail to touch the right parts of the screen. If you touch the candidate's name instead of the box to the right of the name, nothing happens. Again, the red vote button doesn't light up. So a voter could touch each of the candidates names and then walk out. They don't know they're supposed to see lighted green arrows to the right of the name, so they don't realize they've done something wrong. The operator could catch this case, because the machine stays in active mode after the voter leaves. But if the operator is busy and doesn't check the machine after a voter leaves, the next voter could enter the booth and vote before anyone notices.
• The voter could press some of the squares and get some of the lights to turn on, but not press the red vote button. If they walk away before anyone notices, their vote can't be cast. We noticed this happening a couple of times. If another voter entered before we noticed, they might be surprised that some of the lights are on. Then again, the new voter might overlook it entirely. The lighted arrow is a small indicator, so they might not notice it. Or they might think it normal for some arrows to be lighted before a vote is cast.

I think the key usability problem with the Sequoia machine is that it's not clear enough what has to be done to record a vote. The old, mechanical lever machines had a strong feedback system. You had to pull the lever to close the curtain, and pull it again to open it. It was actually hard to get out of the voting booth without pulling the lever. You'd get tangled in the curtain. There's no multi-level system like that here. It's physically easy to leave without recording your vote. The cues are some small lights and a high pitched chirp. These are cues easy for an elderly voter to miss entirely.

Pennsylvania Ballot Initiatives 27 Oct 2008 11:05 AM (16 years ago)

My friends in California have so many ballot initiatives to chose from. On any conceivable issue, you can probably find a ballot measure to oppose. There are two on the ballot for Northampton County Pa. Short summary of this post: Vote yes on both questions.

Poll Worker Training 27 Oct 2008 6:05 AM (16 years ago)

I went for my semi-annual poll worker training session on Sunday. The normal focus of the training session is on operating the voting machines: How to prepare them in the morning, operate them, and record the votes on the flash memory to be delivered for the county count. This year there was an added emphasis on the complications of a presidential election. The most obvious difference with the typical election is that more people come. There will be lines most of the day, primarily because of the time it takes to sign people in.

We were encouraged to be patient. It will be a long day. With long lines, we were asked to keep an eye out for people campaigning inside the polling place or doing anything else inappropriate. The first line of defense here is the election board, but we were encouraged to call the Register with any questions.

Common Cause has a good summary of Pennsylvania election procedures. We were told that the county would provide lots of provisional ballots for polling places and encouraged us to use them. I was happy to hear that Pennsylvania will count a provisional ballot cast at any polling place in the county. If you show up at the wrong polling place 10 minutes before the end of voting, you can cast a provisional ballot and it will still count. My only reservation is that it's more work to cast a provisional ballot. There's more paperwork for the voter and the poll worker. If we have a lot of votes cast this way, it will slow us down.

Provisional ballots sound like a mixed bag. The county collects the provisional ballots, decides whether each one is valid, and counts the valid ones. It will probably take two or three days to count them. The obvious advantage is that anyone not allowed to vote using the regular voting machines can still cast a ballot. If there's a problem and you don't cast a ballot, you can't fix that later. I've read a lot of warnings about provisional ballots. At some level, the warnings make sense. If you can vote on the official machine, that's better. Your vote will definitely be counted--at least to the extent you can trust the machines--but I don't have any idea how they decide which provisional ballots to count and which to toss. I do have some trust in the process:
Howard Erney is the acting chief registrar for this election. I don't know him well, but he runs voter training and often visits polling stations on election day. He seems fair, honest, and conscientious.

I asked about poll watchers and about challenges to voters. The answers here were pretty thin. We had two poll watchers during the primary. One woman was tracking Republication voters--making sure people who said they would vote really came to vote. The other poll watcher came and went. They both stayed while we counted the votes at the end of the day. I did find a summary of the Pennsylvania code on poll watchers that explains things in a fair amount of detail. They can challenge someone's right to vote, but that doesn't automatically disqualify someone from voting. It just makes more paperwork for all involved.

The Pa. League of Women Voters has some more useful information for poll workers. There's a checklist for poll watchers that will work just as well as a checklist for poll workers. I found a really detailed FAQ for Pennsylvania voters on their site, but I don't know if it's current.

Thai Pumpkin Pizza 4 Oct 2008 5:46 PM (16 years ago)

I had a pumpkin and some Thai basil from my neighbor Tina. I found a recipe for Thai pumpkin soup and that formed the basis for a Thai pumpkin pizza that I made tonight. It turned out well. We'll definitely make it again, so I'm recording the recipe here. I made enough for a small 8" round pizza. All the proportions are approximate.

• Cut pumpkin into small cubes, less than an inch on a side. 1-2 cups total.
  
• Puree an inch of ginger, two pieces of lemongrass, and some Thai basil in a blender with a 1 cup of water.
• Strain the ginger / lemongrass paste and save the liquid.
• Dice 3 shallots and saute in oil for about five minutes over medium heat.
• Add the paste and pumpkin to the shallots and saute for a few more minutes.
• Add the reserve liquid, a half cup of vegetable broth, and some nam pla to the pumpkin. Lower the heat and saute until the pumpkin is soft, perhaps adding a little more liquid. Cook off all the liquid until there is just a thick paste remaining.
• Spread the pumpkin / shallot paste on the pizza crust. Add some vegetables. We had slivered snow peas. Carrots would be good, too.
• Add a thin layer of cheese. We used three parts fontina and one part mozzarella.
  

Photo Meme 19 Sep 2008 6:26 AM (16 years ago)

Instructions: Take a picture of yourself right now. Don't change your clothes, don't fix your hair - just take a picture. Post that picture with NO editing. Post these instructions with the picture.

Via: Betz, Kang, Egnor. Also: Haahr.

An Album for Each Year 30 Jul 2008 10:49 AM (16 years ago)

The exercise has been going on for a few weeks in the blogosphere. See Albums Going Steady or My Favorite Album. I was born in 1972, so it wasn't until sometime in the mid 1980s that I had favorite albums that were actually released that year.

• 2007 Spoon Ga Ga Ga Ga Ga
• 2006 Ben Kweller Ben Kweller
• 2005 Fiona Apple Extraordinary Machine
• 2004 Modest Mouse Good News for People who Love Bad News
• 2003 Joe Strummer Streetcore
• 2002 The Flaming Lips Yoshimi Battles the Pink Robots
• 2001 Björk, Vespertine
• 2000 The Delgados Great Eastern
• 1999 Fiona Apple When the Pawn
• 1998 Billy Bragg & Wilco Mermaid Avenue
• 1997 Sleater-Kinney Dig Me Out
• 1996 Tommy Flanagan Sea Changes
• 1995 Bjork Post
  
• 1994 Hole Live Through This
• 1993 Liz Phair Exile in Guyville
• 1992 k.d. lang Ingenue
• 1991 U2 Achtung Baby
• 1990 Replacements (Paul Westerberg) All Shook Down
• 1989 Pixies Doolittle
• 1988 Sonic Youth Daydream Nation
• 1987 U2 Joshua Tree
• 1986 R.E.M. Life's Rich Pageant
• 1985 The Smiths Meat is Murder
  
• 1984 Replacements Let It Be
• 1983 R.E.M. Murmur
• 1982 Violent Femmes Violent Femmes
• 1981 Bauhaus Mask
• 1980 Dead Kennedys Fresh Fruit for Rotting Vegetables
• 1979 The Clash London Calling
• 1978 Elvis Costello This Year's Model
• 1977 Talking Heads Talking Heads 77
  
• 1976 Sex Pistols Never Mind the Bollocks, Here's the Sex Pistols
• 1975
  
• 1974
• 1973
• 1972 Rolling Stones Exile on Main Street

There's a bit of skew caused by which albums I own. And I don't have a great sense for what I listen to from the early 1970s. If I had to pick some runners up, they would be Hüsker Dü Zen Arcade (1984) , Cowboy Junkies Trinity Session (1988), the Minutemen Double Nickels on the Dime (also 1984), and Matthew Sweet Girlfriend (1991).

Python 3000 urllib package 18 Jun 2008 12:58 PM (16 years ago)

I made an svn commit on the py3k branch today--my first commit in a long while.

Make a new urllib package .

It consists of code from urllib, urllib2, urlparse, and robotparser.
The old modules have all been removed. The new package has five
submodules: urllib.parse, urllib.request, urllib.response,
urllib.error, and urllib.robotparser. The urllib.request.urlopen()
function uses the url opener from urllib2.

Note that the unittests have not been renamed for the
beta, but they will be renamed in the future.

Joint work with Senthil Kumaran.

I started urllib2 at CNRI sometime around 1999, based on some experience working Grail (a Python+Tk web browser). I had intended to polish it much more and replace urllib, but I ran out of time. When the PythonLabs team moved from CNRI to BeOpen, we needed to put that code in a Python release or negotiate with CNRI to take the code anyway. So it went into Python 1.6 in its rough form, and I moved on to other projects.

A mere ten years later, we've made some progress. urllib and urllib2 have merged together into a urllib package. The urllib2 code has become the default implementation in the new package. If you call urllib.request.urlopen(), you get urllib2 code.

Senthil is working on a urllib summer of code project to improve the code and documentation further. I'm looking forward to it.

Election Certificate 6 Apr 2008 5:24 PM (17 years ago)

I won a local election a few years ago by a single vote--my own. It was fun to receive the official certification, which has a form to fill out with name and vote total. The form includes the word "votes" rather than "vote(s)," which would have been more appropriate in my case.

Lightning Talks 21 Mar 2008 4:15 PM (17 years ago)

I missed PyCon this year. It was the first time I skipped a Python conference since I started going in 1997. I've been reading reports from bloggers eagerly. One of the larger discussions is about the sponsor lightning talks, which weren't well received. Ted Leung commented that we should thank the Perl folks for the lightning talks, but I think they have a long history at Python conferences.

I ran the first session of short talks at the Python conference in 1997: "Content-wise, anything goes, as long as it is deemed interesting for a room full of Python users and developers, and as long as the content is technical, not marketing material."

I have a distinct recollection of seeing a short talk by Christopher Small at a Usenix conference that inspired our short talks session. He had just a few minutes for the talk and something like 17 slides. He had someone turn the slides for him at regular intervals and he tried to keep up. It was really entertaining. (I would have guessed it was at the 1996 OSDI WIP session, but he's not on the list of speakers.)

The first speaker at our short talks session with Scott Deerwester, and he agreed to use the same presentation style as Small. I turned his slides for him, whether he was ready or not. He was a good sport and I think it set a light tone for the rest of the session.

We didn't have short talks the following year and didn't have a Python conference at all in 1999. David Beazley ran the short talks session in 2000. (Andrew Kuchling noted the short talks in his conference summary.) The Python folks started calling them lightning talks in 2001 at the 9th Python conference. David Ascher ran a session called lightning talks at OSCON in 2003, too.

I think the name lightning talk is due to Mark-Jason Dominus. We exchanged some email about the Python short talks long ago, when he was planning the first lightning talks session at a Perl conference. I've never seen a Perl lightning talk session, so I don't know how they compare to the Python short talks / lightning talks. I think they deserve a lot of credit for popularizing the idea.

We've always tried to have fun with the lightning talks. It's a good place for outrageous opinions or quick demos. We've gotten less and less formal about it, too. The first few session required you to sign up in advance and had a full schedule. Later, we let people sign up on the spot and made up the schedule as we went along.

The lightning talks have become one of the most popular sessions at the recent PyCons. They've been scheduled with nothing opposite them (like keynotes, unlike regular talks), and there have been several sessions of them.

Dave Weissman needs a haircut 24 Jan 2008 10:50 AM (17 years ago)

He may argue the point, but I think he is wrong.

Update: mreid agrees. kang agrees.

Beautiful libraries 18 Sep 2007 10:13 AM (17 years ago)

I found this love letter to libraries via ResearchBuzz. I've only been to a few of these libraries myself:

• The Peabody Library in Baltimore. I lived a few blocks away.
• The Long Room at Trinity College Dublin. I lived more than a few blocks away, but not too far.
• The public libraries in Boston and New York
• Not on the list, but others I've enjoyed: The Widener Library. The Hayden science library at MIT is not pretty; it looks like a cheap motel. It does have a wonderful view of Boston and the Charles River.
  

McNamara, Unions, and Labor Costs 9 May 2007 6:32 PM (17 years ago)

Paul Graham recently posted An Alternative Theory of Unions that compares union workers with servers--in particular with the notion that Internt Bubble companies overpaid for servers the way manufacturing companies overpaid works in the 1950s. He picks car manufacturers as a specific example:

If you looked in the head of a 1950s auto executive, the attitude must have been: sure, give 'em whatever they ask for, so long as the new model isn't delayed.

I don't agree with the argument in general, and I think this particular suggestion is just plain wrong. Robert McNamara who was responsible for financial controls and planning for the Ford Motor Company in the 1950s, ultimately becoming president in 1960. McNamara was well aware of the cost of every part of producing a car and worked hard to keep costs down at Ford.

I've been reading David Halberstam's brilliant book on Vietnam, The Best and the Brightest. He has an interesting discussion of McNamara's relationship with labor unions at Ford. He suggests that McNamara, a liberal former Harvard professor, was sympathetic to union concerns but more concerned about cutting costs.

McNamara was one of the people behind the Ford Falcon, a smaller, cheaper Ford car that was a big success. The Wikipedia article on the Ford Falcon observes that towards the end of the 1950s American cars were becoming increasingly expensive because of wage inflation.

I don't know much about manufacturing in general, but the analogy does not seem to apply in cars. Labor was expensive for carmakers in the 1950s. They seemed to be aware of the costs and tried to control them.

If anything, the earlier generation of Ford leadership liked unions even less. Halberstam writes,

Under Henry [Ford] senior and Harry Bennett the policies of the company were singularly primitive. The public was a problem, the unions were a problem, the bankers were a problem.

Update: Some of the comments on Paul's article at Marginal Revolution make similar points.

Armadillo Migration 12 Apr 2007 7:27 AM (17 years ago)

There's no reason to post about armadillos today. Many years ago, I read an entertaining article in the Washington Post about armadillos. I remembered that the armadillos could be expected to survive in Maryland and Pennsylvania, but I didn't remember how slow the northward migration was. I foolishly hoped to see an armadillo in my backyard one day.

Tara and I found the original article in Google's news archive last night. It says

In another century or so, they could reach the White House as they continue their northward migration.

It will be a long time until we see them in Easton.

The article quotes Lynn Robbins on the subject. His co-author Joshua Nixon has a short overview page on armadillo expansion.

Flight delayed, bugs fixed 27 Feb 2007 9:35 AM (18 years ago)

My original flight from Dallas back to Newark was cancelled because of mechanical problems with the plane. I got bumped from a 10:30 flight to a 2:45 flight, which left me a few hours for hacking and socializing at the PyCon sprints this morning. I had breakfast with Jeff Elkner and caught up on his software and teaching projects, and I got to finish some bug fixes in typeobject.c that I had worked on yesterday. I find it very satisfying to fix an Armin Rigo crash bug. It's usually an accomplishment just to understand it.

nonlocal implemented 26 Feb 2007 9:52 PM (18 years ago)

I'm going to love Python 3000! Thomas Wouters and I implemented PEP 3104 tonight. It fixes a wart in the original nested scopes implementation that I did in 2001. In that version of nested scopes, we did not allow names defined in one function to be rebound in an enclosing function. It was impossible for the compiler to distinguish between an assignment that creates a local and an assignment that rebinds. Python 3000 will fix this using the nonlocal statement. I hate the name nonlocal, but no one has thought of a better name. Ka-Ping Yee wrote PEP 3104 and provided an exhaustive list of alternate names,

The code itself was quite simple. The only changes in compile.c were trivial. The symbol table needed more changes, because it had to recognize a new kind of declaration and propagate that information to the compiler. The symbol table uses a bit-field where a handful of the bits are used to represent the scope. I spent a lot of time scratching my head until I remember that I needed to increase the width of mask used to extract the scope-related bits. I think I'm not happy with the bit-field representation.

Pete Shinners and Neal Norwitz reviewed code and helped think of tests cases.

That wraps up a fun day of sprinting for me. I also closed several bugs and spent a few hours pouring over typeobject.c to fix some crasher bugs that Armin Rigo reported. I have a fix for one of them, but I want to do a little refactoring before I check it in. Next week, perhaps. I'm flying home first thing in the morning.

Bug of the day 26 Feb 2007 3:18 PM (18 years ago)

Mike Verdone found a great Python bug today. It's an interaction between two features.

• Name mangling. If you use an identifier with two leading underscores in a class, the name is mangled to include the name of the class as a prefix, except if the name also ends with two trailing underscores. In the class Spam, the identifier __eggs becomes _Spam__eggs. Note that the language reference says that mangling applies to any identified
  
• Packages. You can import a name like A.B.C. This name describes a module C in package B inside package A. Note that A, B, and C are identifiers, so if one of them has two leading underscores they should be mangled. import __A.B.C should technically be turned into import _Spam__A.B.C if it occurs inside class Spam.

One detail of the implementation is that the compiler represents the sequence of identifiers in a package name as a single string. The compiled code represents the dotted name as a single string argument to a bytecode instruction. The implementation of name mangling checks for strings that start and end with double underscores. Therein lies the problem. The string "__A__.B.C" starts with two underscores, but does not end with two underscores. It is mangled to "_Spam__A__.B.C."

How old is this bug? It is present in Python 1.5.2 and was probably present in the original release of Python 1.5. The bug is about nine years old.

We're going to do a quick fix: Change name mangling so that it does not mangle names with dots in them. This will fix the __A__.B.C case, break the __A.B.C case, and leave broken the A.__B.C case. It is likely that we will change the language spec and say identifiers in import statements are not mangled.

locals() and free variables 26 Feb 2007 8:17 AM (18 years ago)

I've been struggling with some odd corner cases of free variables and eval/exec in Python.

There is a long-standing bug report that you can't access free variables when you created a nested code block with eval or exec--for example, putting a lambda in an expression passed to eval. It works if the variable happens to be free in the text of the function containing the eval, but only then. The bug report claimed that Scheme worked differently, but I don't think Scheme provides a way to capture an arbitrary environment to pass to eval; it only provides access to top-level environments.

The name locals() is misleading. It returns free variables as well as local variables. The name suggests that this behavior is wrong, but it is necessary to make exec and eval work with free variables. The function returns the names visible in the current scope.

The discussion here is actually about much more than locals(). The same basic issues arise when you use exec or import * in a block or when you use the debugger or some other trace function installed via sys.settrace(). In all these cases, we extract variables in a dictionary to support introspection.

There is still a problem with locals(), which results from the use of locals() in debugging. The implementation makes sure that changes made to the locals dict are reflected in the running program in many cases. In old versions of Python, local variables were stored in a dictionary and locals() returned the actual dictionary. This feature is useful in the debugger. When the implementation changed to use a simple C array instead of a dict, the interpreter arranged to copy variables back and forth between the dictionary and the array when the debugger was used. (This behavior is also needed to make features like exec and import * work.)

In many cases, it is fine to copy free variables into the dictionary returned by locals. If changes are made to those variables, they can be written back to the appropriate part of the closure rather than creating local variables that shadow the free variables.

Class namespaces pose a serious problem in the current CPython implementation. The class stores local variables in a dictionary. When the body of the class finishes execution, this dictionary is passed to the class constructor. Keys in the dictionary become attributes of the class. If you call locals(), free variables could be copied into the dictionary for access in the debugger or introspection. But if they are copied, they will become attributes of the class, which was not intended. It's a messy problem, because it is possible, though inscrutable, for
the same name to be used for a free variable in a method and a class attribute; they have the same name, but they refer to different bindings.

For now, we are fixing this in Python 2.x by omitting free variables from the dictionary returned by locals() when locals() is called in a class block. This makes introspection more difficult but prevents locals() from polluting the class namespace.

What are some other solutions for the class problem? It might be possible to return a copy of the class dictionary with the free variables added. Changes to this dictionary are written back to the real dictionary or free variables when you run in the debugger. It wouldn't be possible to reflect all changes to the dictionary; it would only work in contexts like debugging or using exec. I'm not sure if that would be too confusing.

We could also make two different functions, where locals() and vars() seem like reasonable names. The locals() could return the actual dictionary object for classes, without adding the free variables. The vars() could return a dictionary will all variables, but it would be a copy. Then client code could get whatever they wanted.

PyCon Pictures 25 Feb 2007 10:26 PM (18 years ago)

My first Python conference was almost ten years ago, at a hotel in downtown San Jose. Guido introduced Python 1.5 that year, and Jim Hugunin introduced JPython. I ran the very first lightning talks session. Ten years later, there are still a lot of familiar faces, but the conference is much bigger. The community is larger and their interests more diverse. The keynote speakers were variously engaging and entertaining, probably equally Randy Pausch's talk in 2000.

I attended a few talks, fixed a few bugs, and got a chance to re-engage with Python development. I have had little time for Python since 2.5 was released and the compiler rewrite finished. I'm looking forward to a full sprint day to get some code written. I had hoped to fix some obscure bugs, but Armin Rigo's bug reports were too obscure to get to the bottom of with just an hour's study.

A small group of python-devers and Googlers had an over-the-top dinner at the Mansion on Turtle Creek.

I've collected a few PyCon pictures over the week, snapshots of friends, speakers, and other Pythonistas. Update: I added a few pictures from the sprints on Monday (2/26/07).