Category Archives: Peter’s Stuff

Things Peter is doing, interested in, or otherwise feels like posting

An Expensive Toaster

A few years ago, I got tired of a toaster that didn’t bother popping up before the toast was charcoal, so I got on-line and ordered a commercial-grade toaster from some restaurant supply store.  Based on the presumption that restaurants couldn’t afford to have a toaster that didn’t work right, I hoped to have a reliable appliance well into the future.  Unfortunately, that hope was unfounded.  Within two years, the toaster quit working.  You could hold the handle down and the heating elements would do their thing, but it wouldn’t stay down.

Isaac, being one of the bigger fans of toast in this house, rigged up an elastic band that would hold the handle down, but it wasn’t long until this arrangement produced it’s share of carbonized bread.  Unfortunately for Isaac, I don’t eat a lot of toast anymore, so I didn’t really have much of a motivation to replace it.  However, a few weeks ago I got a bug in my ear, went out to the garage for some tools, and tore down the toaster to see if I could figure out what was wrong.

The toasters I’d taken apart in the past (yes, I’ve disassembled one of almost everything at this point in my life) all used a bi-metalic strip in a mechanical assembly to hold the toast down until it heated up enough and bent far enough to release the mechanism.  All in all, it’s a pretty simple setup, if a little unpredictable when trying to set the toaster for the perfect toast. I figured something had been bent or broken in this kind of an arrangement, and that I could probably adjust or repair it.  If not, what difference did it make?  The worst I could do was to ruin an already broken toaster.  However, when I got it apart I realized my original assumptions were wrong.

This high-falutin’ hoity-toity commercial toaster used a digital timer circuit to control how long to leave the toaster on.  This, as luck would have it, was a good thing.  I’m reasonably good with mechanical stuff, but my bread-and-butter has for years been electronics.  I pulled out my well-worn but still functioning test equipment from the days when I actually got paid for fixing things and went to work.  Within a few minutes I’d reverse-engineered the simple circuit and identified the bad part.  Easy, I thought to myself, I’ll just add this part to my next Digikey order for one of my other geek projects.

As it turned out, Digikey doesn’t carry the integrated circuit I needed.  Neither do Mouser or any of the other parts-houses I’ve used before. The defective part was made by a Chinese semiconductor company and only available from China.  In fact, the only place I could find to order it from without having to read Mandarin (or whatever language it was) was aliexpress.com, and it would take 2-10 weeks to arrive.  The good part was that it would only cost me about $5.00 for 5 of the things including shipping it over on the slow-boat.  I placed my order and sat back to anticipate the part’s arrival.  What I’ll do with 4 extra “toaster timer” IC’s I’m not sure.

The parts arrived yesterday.  Compared with Amazon, the shipping times leave a little to be desired,  but I had the part in-hand so I went out to the garage and went to work without grumbling too much.  I soldered in the new component, tested the toaster, and put it all back together pleased with a job well done.  However,  no sooner had I assembled it than I realized it wasn’t working.  While it had been working on the bench while disassembled, it seemed to have a different opinion about proper behavior when fully assembled and on the kitchen counter.

I spent the next three hours disassembling, testing, tweaking, reassembling, testing disassembling, etc…  Any time I put it all back together it wouldn’t latch the handle down.  As it turned out, somewhere along the line, someone pushed a little too hard on the handle, and it bent the assembly, almost imperceptibly, but significantly.  Any time the whole toaster was assembled, the case would interfere with the handle and prevent it from fully engaging the locking mechanism.  It took forever to get everything straightened out and working reliably.

So, this gets down to my problem… I was seriously considering tearing out the electronics and programing one of my Arduinos to control the toaster.  What kind of an idiot would do that?  The alternative wasn’t much better.  Instead of re-engineering a solution, I spent an inordinate amount of time fixing something that, in the end, is worth no more than half an hour of my time if I were to bill out at an hourly rate.  I should have stopped before I even took it apart the first time, little lone the time and energy spent actually fixing it.  The fact that I was actually cogitating re-engineering the thing to work with a custom-programmed micro-controller speaks volumes.  Normal people would have just thrown the dumb thing in the trash and gone to WalMart for a new one.

Recreational work

Stop, I'm told, and smell a rose.
Pause and take a break.
So I comply.
The smell offends my nose.

Why don't you do what others do?
I'm asked without words.
But I'm not them.
Must I pretend to be like you?

What's wrong with loving work?
Both the process and results?
Rest is wearying.
But labor refreshes and refuels.

Geeking Out

Since we moved back into a city and left behind a mostly rural life I’ve had to adjust my hobbies to the new environment.  I don’t spend much time planting fence posts, the only animal I take care of now is an old and grouchy dog, I don’t have old or worn out farm equipment to rebuild or repair, can mow my lawn with a weed whacker, and couldn’t fit a car in the garage to work on it if I had to.  This has been a hard transition, but it has given me an excuse to reengage with several hobbies that have lain dormant for quite a while.  One of the things I’ve restarted has been playing with electronics.

Back in high school I spent a lot of time in the electronics shop designing and building various projects that included a simple television jammer, an all electronic Christmas tree that could flash lights in various sequences, and a custom built clock that was crazy enough that it made airport security really uncomfortable when I took it to Kansas City for a national competition (after x-raying it, they  made me disassemble it and show them it wasn’t a bomb).  My brother Tolon and our neighbor Mike have tons of stories about the crazy stuff we did with electronics growing up, so needless to say, it was a big part of our adolescence.  However, once I left college and the part-time television repair job I had, electronics mostly fell by the wayside.   It wasn’t that the interest faded.  It was really just that I had a long list of competing priorities and interests.  Besides, I don’t have much use for the kinds of simple projects I could build at home.  While flashing lights are cool in some respects, it doesn’t do much for me once it’s built.  I have a higher standard of applicability for projects these days.

This last Christmas, however, I bought Isaac a robot car that used a raspberry pi for the controller.  Unfortunately (or not, depending on how you look at it), the cheap Chinese version I bought came with code that was so bad that I spent my Christmas break learning Python and writing new code from scratch.  I hadn’t done much coding for a long time, and in the process of making a functional software set for the robot I remembered how much I like that kind of a puzzle and challenge.  Another thing working on Isaac’s robot did was get me back looking at electronics hardware design and thinking about the software/hardware interface in ways I hadn’t before.  I have been a Linux geek for over 20 years, and the possibilities of a super-cheap, super-small Linux-based computer with exposed low-level I/O ports really piqued my interest to the extent that I actually had a spell of insomnia while a whole host of ideas for things I could do with it flooded my mind.

It didn’t take too long for me to realize that the best way to get sleep would be to pick a project and get to work on it.  As I thought about it, I kept coming back to a project I’d built years ago and decided it was time to update it and build it again.  The original project was nothing more than a Mr Coffee that I stripped the electronics out of and repurposed to turn on a light at a given time every day.  This light woke me up every morning for several years, but it was bulky and ugly, and the way the light turned fully on instantaneously was rather jarring.   In an effort to fix this shortcoming, about five years ago I designed and built an analog circuit to gradually turn on a bright LED lamp.  Unfortunately, Liz hated the bright white color and couldn’t stand the fact that the LEDs weren’t diffuse enough.  They were rather blinding if you looked right at them.  I had to retire the light and Mr Coffee electronics.

In thinking about the possibilities of a super cheap computer with easy to use hardware interfaces, I came to the conclusion that I could easily build a control circuit to digitally dim a three color LED.  I decided to build a raspberry pi based project that would mimic both the brightness and color of a sunrise at any time I wanted to wake up.  Because it was software driven, I could tailor both the brightness and color to match a sunrise from start to finish.   I would also be able to set it up so that I could program different alarms for weekends or any other unusual day.  Not only that, but I could even program it to do any kind of goofy rainbow-colored light show if I ever got the inkling (but I have to admit that the odds of that are kind of low).

Now, I haven’t made custom circuit boards in almost 20 years, and I haven’t written much code since I left grad school.  A sane person might hesitate to jump into what amounts to two simultaneous complex projects, and you might think it would at least have given me pause.  But no, true to form I jumped in.  I scoured datasheets from several electronics vendors, ordered materials for etching circuit boards, and began learning Eagle CAD.  Before too long, I had a schematic complete.  Not long after that, I had a circuit board layout.  A few failed attempts later I had a complete board and was on my merry way writing code to see if it would all come together the way I intended it.

The top board is a Raspberry Pi Zero W. Beneath that board is the first completed controller board I built. The piece hanging off the right side is a real-time-clock module that keeps time when the power is off. The LEDs, and the circuit board with the LCD panel and input buttons aren’t connected in this picture.

After fumbling around with Python and learning some of it’s ‘features’ the hard way, I managed to make the LCD and buttons I bought work as a menu of sorts, and got it to start the sunrise sequence on-schedule.  Pretty stoked about my progress, I plugged it in near my bed and set it to go off the next morning.  Unfortunately (or perhaps fortunately) I woke up a few minutes before the alarm and was watching it as the first blue-gray light started to come on.  Almost immediately, I realized my design wasn’t going to work.  At low brightness, the light flickered enough to really bother me.  I wanted to understand why, and what I could do about it.

As it turns out, one of my day-job responsibilities is setting design criteria for safety critical systems, and I had spent a good deal of time recently getting familiar with what are known as “real-time operating systems” and their role in safety-critical applications.  In any modern computer, multiple processes (programs) compete for processor time.  The operating system will hand control to a process, and how long that process has control before the next process gets to do it’s part is not tightly controlled.  For example, if you’ve ever been working on a computer that suddenly ‘hiccuped’ for a second and then caught up with you, you have been the victim of a non-real time system.

For routine applications, a slight hiccup that results from a process taking too much processor time isn’t all that important.  The process will eventually finish and the computer will catch up without any dramatic consequences.  However, if that computer were flying a plane or guiding a missile, even a slight hiccup could result in a crash.  For these situations, a special type of operating system is put in place to make sure that the important stuff always gets the processor time it needs, and only the unimportant stuff occasionally gets stuck with a hiccup (if at all).  Everything runs on a heartbeat, and important stuff gets done every heartbeat, every time.  When done properly, this kind of a system ensures that there aren’t any hiccups on things that matter.

So, how does this apply to my stupid little project?  First, it’s fundamentally difficult to efficiently dim an LED the way you would dim a conventional light bulb — especially using a digital controller.  Instead, to dim a high power LED you turn it on and off very rapidly, and our eyes average the intensity naturally.  To make the light brighter, you leave the light on for a longer percentage of the time.

Well, as it turns out, the code that determines how long the LED stays on works by deciding every few microseconds whether to turn the light on or off.   Because the ultra-bright LED’s I used are so bright, when I want them to be very dim the lights are only on for a few of these cycles before they turn back off again.  Even small changes in how long the lights are on in that case make a big difference in how bright the light looks.  The way this thing is set up, it turns the LED on and off 200 times a second (each cycle is 1/200th of a second) to make sure the human eye can’t see it flickering.  To make the sunrise work, I break that 1/200th of a second into 1024 chunks (why not just a round 1000 you ask… 1024 = 2^10, and computers do most things best with powers of 2).  When the light is at it’s dimmest, it’s on for 1 chunk (about 1/200,000th of a second) every cycle.  When the light is all the way on, it’s on for the full 1024 chunks and never turns off.

When the light is at it’s dimmest (where I start it in a sunrise sequence) even a five microsecond delay (about 100000 times faster than a blink of the eye) will cause the light to stay on twice as long as it should and change the brightness by a factor of two.  Small glitches make a big difference.  As the code runs through it’s loop deciding when to turn the light on or off, other processes randomly run a little long, and the decision to turn off the light is delayed just a bit.  Because the raspberry pi isn’t running a real-time operating system, there is competition for processor time, and that shows up as a flickering light that flickers worst when it’s on very low.  Because I’d just been through the wringer working through the real-time operating system stuff at work, I instantly recognized the situation for what it was.

Unfortunately, this realization meant that my design was fundamentally flawed.  I had relied on the pi for timing, and even using the most optimized code I could find to run the timing, the light still had a noticeable flicker.  Did that stop me?  If you are reading this, the odds are pretty good that you know a fair bit about me already, and could guess that I don’t quit that easily.  One solution would be to implement the code in a real-time operating system.  However, I’m not really an expert on those systems, and I don’t believe they’ve ported real-time linux to the pi.  The answer would have to be a hardware-based timer for the lights, and the best part is that I already knew where to go to find one.

Something I learned while working with Isaac’s robot car was how servos work (the small motors that steer the car and point the camera).  They work by generating a pulse with a very specific width proportional to the turn angle at regular intervals, just like the way I was trying to control the lights.  Robot designers don’t use the I/O pins on the pi to control servos for the same reason it didn’t work well for the lights — flickering, but in the case of servos, they don’t flicker, they jitter back and forth.  If you’re using the servo to control a remote-control plane for instance, that jitter would make the plane uncontrollable and result in a crash.  In Isaac’s car, the designers had actually re-purposed an LED controller with built-in timing to generate the servo pulses, and I was already familiar with how that controller worked and how to program it using the pi.  It would be an easy answer.

Frustrated, but unwilling to give up, I pulled open the schematic, deleted the control lines that I had been using, and added in the controller.  A few days of spare-time later, and I had a new board layout ready to process.  Thankfully, when I originally ordered parts for the first board, I had planned for the probability that I’d ruin several boards and had ordered plenty of extra resist film and copper-clad board.  I was also willing to bet I’d break something or need to re-build another board, so I had spares of all the parts.  I etched and solder-masked another board and built it out with everything but the new controller.  At this point, I’m waiting to order the new part and am looking forward to getting the code written and version 3 up and running (version 1 died before the first hardware met the real world).

Version 3 of the board. The new controller IC goes where the copper pads are. I’m pretty confident this one will work to my satisfaction.  The three big black things are the drivers for the red, green, and blue LEDs.  The cluster of components towards the bottom of the board are a 5V power supply to power the pi and all the other electronics.  I also added a real-time-clock module that keeps track of the time when the power is out, but that’s on the other side of the board.

At this point, if I were to add up the number of hours and the money spent on parts and processing stuff, this would definitively be the most expensive clock I’ve ever owned.  To make it worse, I’m looking at probably 20 more hours of coding to fix bugs in the current code and integrate the new controller (I’m not particularly efficient with Python).  Given my going labor rate, I can’t afford my own time.  But then again, it’s not really about either the cost or the time.

Because I Can

I'm told it'd be better and cost less,
If I hired the experts to do it.
They reason true.
I know.

My time's too costly for stuff like this,
I should just pay someone else.
Again, they're right.
I know.

But money and time aren't the point,
I do it myself 'cause I can.
Joy has value.
I know.

 

Caught it…

The rumble of wheels on gravel
I prick my ears and take position
For years I've tried diligently but failed
Today is the day -- I will catch it today
I launch with all the power in me
It draws near and I lengthen my stride 
Barking fiercely and closing the gap
A mouthful of rubber -- thrill of success
Then searing pain and darkness close in
As I ask myself why I wanted this.

 

Why do I write?

A little while ago somebody asked me what motivates me to write.  I’ve thought about that off and on for a long time, so you’d think I’d have a pretty solid answer by now.  I don’t.  At least, not really.  It’s a case where the real answer is somewhat amorphous and changes shape from time to time depending on the circumstances. When I think I have a relatively complete answer, something around me or in me shifts just enough to alter the answer in substantive ways, and I am left with a hole that hasn’t been filled in yet.  That said, there are some motivations that I consider enduring.  They have remained consistent and applicable throughout my memory, and I expect them to remain so indefinitely.

First, and fundamentally, I write to please myself.  This sounds to me quite selfish, but I believe there is probably an element of selfishness at the bottom of almost everything we do of our own free choice.  At the end of the day, I find writing generally rewarding as I pull together thoughts and memories to fashion them into something that I hope is coherent and interesting.  Telling a story or writing a poem is a lot like building a project where I have a design in mind, select and collect the pieces, shape and fit them together in a manner unique to my intent, and assemble something that I find useful or pleasing as the end result.  I get great pleasure in stepping back at the end of a project and seeing a result that I can be somewhat proud of.   Whether it’s a physical structure, electronics project, or string of words on a screen or paper, I like to see a finished result that reflects the care and effort I put into it.  It’s satisfying in a way that my daily employment isn’t.

Another factor that motivates me when I write is the ability it affords me to organize, analyze, and assess complex issues.  I have a wide range of ideas and ideals that are shaped by a vast array of life experiences, but the linkage between those experiences and the beliefs and ideas is often buried and uncertain.  When I sit down to write about those kinds of ideas it give me the opportunity to analyze my beliefs and identify many of the underlying factors that they are founded on.  When I write, I can more clearly identify the linkages between and lineage of ideas, and can take the time to choose how to show the connections and deeper aspects.  I don’t, however, generally write about fundamentally deep ideas and make them widely available.  Those writings are more often than not reserved for me alone.

There are times that I write because I can be more precise about a message I mean to communicate.  I’ve often heard it said that written language is one of the weakest forms of communication because you lose much of the context surrounding the message.  While that is generally true, I don’t believe that is universally the case.  Hastily written messages are, in fact, dangerous because they can be very easily misinterpreted. Non-verbal queues, inflection and intonation, and immediate feedback are all lost due to the delay and separation that occurs when we communicate in writing, so a poorly crafted thought can lead to amazing misinterpretations.  However, I’ve found that there are a wide range of topics for which verbal communication is much more dangerous than writing.

Contentious topics or complex issues require great thought and deliberate approaches that are easily screwed up when responding to someone in the heat of the moment.  Writing on these topics allows me the time and opportunity to analyze the messages being sent, evaluate them against my purpose in communicating, and adjust them appropriately before the intended audience has received the wrong message.  Taking time to write out my thoughts also allows me to analyze the concepts, evaluate the supporting arguments, and ensure my position is well founded.  Carefully crafted writing, while missing the nonverbal elements of communication, is uniquely well suited for dealing with thorny, contentious, or complicated issues.

Sometimes I write because it is easier for me to put strong emotions or difficult topics into words when they are written.  I find certain things very difficult to speak about with a steady voice and a rational mind.  I often use poetry, in particular, to touch on these things I can’t really express otherwise.  The ability to address these kinds of emotions without directly speaking to them and in a form that can mean something completely different to each new reader has drawn me to poetry, especially when I’m having difficulty communicating in other ways.

The last reason I’ll touch on is probably the most fundamental and enduring one.  I write to leave a piece of me behind.  Much of what we know about history comes from writings left behind by those who went before.  In our modern world, people have shifted to less and less durable forms of communication.  By the time my children are having children, much of what I experienced will be lost to modern memory if it isn’t recorded somewhere.  The stories of my childhood won’t be there to entertain and educate my children, grand-children, and great grand-children if I don’t write them down.  I want my progeny to know who I was so they can understand a little of where they came from.