ACS Alive and Well

Yee-hah! After 15 years in the space biz, finally something worked!  About time.  Kudos to the amazing team of people who made it happen, ending with Megan McArthur controlling the arm, Drew Feustel fetching the tools and parts from the storage bins, and John Grunsfeld pulling out the old and clicking in the new.

The Advanced Camera for Surveys is what took most of the really cool pictures you’ve seen from the Hubble Space Telescope in the last few years.  Then it broke.  The A side power supply for the CCD Electronics Boxes (CEBs) lost its 15V supply in the middle of 2006.  They switched to the backup (B) power supply and kept on running.  But then at the end of January 2007, the B side power suppy fried itself totally, in what must have been a very exciting flash if someone had been there to see it.  It was drawing close to a kilowatt for at least 10 seconds, and we’re basically talking a computer power supply here.  The pressure sensors inside HST registered some gas at the time, so something toasted itself but good.

Too close to the next mission to replace the whole ACS (the first one took 5 years or more to build), but a certain scientist/engineer named Dr. Ed Cheng thought we might could fix it.

Some background:  ACS comprises three cameras actually: Wide Field Channel (WFC), a 16 megapixel CCD camera with very low noise, which took most of the cool pictures; High Resolution Channel (HRC), a 4 megapixel CCD with smaller pixels, which was less used; and the Solar Blind Channel (SBC), even more specialized, and less used still.  SBC was still working, but the WFC and HRC electronics boxes ran off the same power supply, thus they were both no longer working.

So.  The plan:  Remove the circuit boards from the WFC CEB and install new ones which take their power from an external plug.  Then put in an external power supply and plug it in.  Brilliantly simple!  The new circuit boards connect to the CCD detector and the rest of ACS the same way the old ones did, through the original motherboard, which stays in.  Simply brilliant!

But what about the HRC?  It’s still connected to the broken power supply.  Well, we could replace it as well, except!  The spacewalking (we like to call them EVAs, or Extra-Vehicular Activities) timeline was already full, installing a new spectrograph and a new camera, and fixing another spectrograph, as well as replacing gyroscopes and batteries.  So the clever idea was to provide a second set of voltages in the power supply, and apply that to the power input pins of the new WFC CEB.  Then the power can run back to the old dead power supply and over to the HRC CEB.  If the old power supply failed by shorting at the output, then the new supply would be shorted too and not work.  But any other failure of the old supply would allow us to fix HRC without replacing its CEB.

So we have to remove four circuit cards from a the Wide Field CEB, a place that was never meant to be messed with in space.  First step is to cut the EMI grid over the CEB.  Yes, cut.  Because it’s faster than removing all the screws, that’s why.  So there was a special tool for that.

John slides the grid cutter into place, and starts turning the cutting bolts with the Mini Power Tool, counting turns.  “One, two, three, click.  Four turns on #1”.  We can’t hear the click as the aluminum grid shears through (in space, no-one can hear you click), but he can feel it in his hand, so he announces it to us.

11 of 12 grid legs are sheared through.  The final one he has to wiggle a bit to break it.  Into the trash bag quick, without cutting his glove on the sharp edges, and on with the cover removal task.  32 small (4-40 size) screws to remove, without letting them drift around the interior of the telescope.  For this, there’s a special plate that clips onto the CEB, with a clear top.  The top has holes large enough for a screwdriver bit, but smaller than the screw heads.  Also two sticky pads that stick onto the CEB cover.  Then when all the screws are out, the CEB cover comes with, trapping the screws.

John Grunsfeld removes a card from the ACS CEB.

John Grunsfeld removes a card from the ACS CEB.

Now we “just” remove the cards.  Each card is in a slot, locked in place by some wedge clamps called card-loks.  So we have to loosen the card-loks with an allen wrench, and slide out the cards.  Simple, yes?  Except that the cards tend to get a little jammed.  Oh, and space suit gloves are too clumsy to reach them, plus the whole sharp edges thing.  (Ever pulled a board out of a tight spot in a computer?  Remember all those little cuts on your fingers when you were done?)  So there’s a special Card Extraction Tool, which grabs each card by the little L-bracket on top and pulls it out by turning a knob.

Here’s one of the cards coming out, as viewed by John’s helmet cam.  You can see the blue and silver Card Extraction Tool on the right edge (the top of the card).  The tool is Indexed, so it can pull any of the four cards, thus making it the ICE-T.  Hey, have we got good acronyms or what?  The ICE-T can put 400-500 pounds of force on the card, in case it’s really stuck.  None of them were.  John made it look easy.  The 4th card was expected to be a little tight against a strut in the telescope on the way out, so we gave him a little tool to hook the card and pull it outta the way.  That worked perfectly too.  “I don’t think brain surgeons go ‘woo-hoo’ when they pull something out.”  That’s okay John, we’re all woo-hooing in the control room, you go ahead and join in.

The nominal plan was to pull two cards, install a temporary cover, and come back on the last EVA to finish the job.  Well, until the Science Instrument Command and Data Handler broke a week before the mission.  That delayed the mission from last October to now, as well as pushing part two of ACS repair clean off the timeline.  So now it was either finish on the first day (EVA day 3) or get part 2 reinstated on day 5, which would only happen if STIS repair didn’t work on day 4.  Very complicated.  But after two boards were out, they were maybe 50 minutes ahead of the planned timeline.  So of course they got a go to finish the task.  The Mission Operations Manager (MOM — is that a great acronym or what?) asked us if we thought they should go ahead, but with a smile on his face, cause they’d already given the go.

So all the cards came out, and meanwhile Drew was desperately trying to catch up to his part of the task, fetching the replacement parts from the stowage in the cargo bay.  (Keeping in mind that ‘desperately’ doesn’t mean what I think it means.)  But John was so fast on his part that at one point he got to rest while Drew caught up.  You can only move around so fast in the cargo bay.

The CEB Replacement slid right into the open space “like putting a book on a shelf”.  We’d been using that phrase since the first time we tried a mockup in the high fidelity mechanical simulator at Goddard, so I was glad to hear Mike Massimino use it while calling out the steps on the checklist from the shuttle mid-deck.  A couple of screws to clamp it in place, and one screw each to insert the 4 new boards into the connectors on the old motherboard.

We had put the new boards in a nice enclosure to make it easier to insert them all, and it worked like a champ.  All those weeks of testing, with shims to make sure it would work if the connectors weren’t perfectly aligned, at hot and cold temperatures, in something like 50 different combinations of possible mechanical misalignments, finally paid off.  Each board has a 320-pin connector on it, and while not all pins were used, there were plenty of them which would fail the repair if they didn’t mate correctly.  But everything went in nice and easy.

Then the new power supply was mounted to a hand-rail, and hooked up to a splitter cable that pulled power from where it feeds into the ACS proper.  Again, months of design and weeks of testing resulted in two seconds worth of sliding the box into place and flipping the clamps closed.  Done!  Just like that.  Hook up the output cable from the new power supply to the CEB-R, and the astronauts’ job is complete (modulo some cleanup).  From the control center, we ran the Aliveness Test, quickly checking as many of the pins on the connectors as possible.  Perfect.

The Aliveness Test was maybe going to have to wait for ACS to warm up some, since everything is cooling down while the Aft Shroud doors are open.  But the task went so damn fast that everything was still warm and we could do it immediately.  Yeah!

Drew gets ready to close the doors

Drew gets ready to close the doors

While we ran the AT, Drew and John packed up all the remaining tools and closed the doors.  Back into the airlock, and the total time for the EVA: 6 hours 36 minutes.  Or 6 minutes longer than the nominal time, even though they did an extra hour and a half of work.  And the 6 minutes over is less than the time they spent rebooting John’s mini workstation when the display wasn’t working at the start of the day.

Oh yeah, and before they did ACS repair, they had already pulled out COSTAR (the corrective optics installed in the first servicing mission, no longer needed since the instruments all have their own corrective optics nowadays) and replaced it with the Cosmic Origins Spectrograph.  Not a bad day’s work.

After a short wait while the COS Functional Test was run, we got the results from the ACS-R Functional Test.  The sad news is that the HRC was not repaired.  The original power supply failure was indeed a short circuit, which means our clever back-powering scheme is shorted out as well.  (As noted above, we could instead have replaced the HRC CEB as we did with the WFC, but there was not enough EVA time to do it, nor was there enough calendar time to build and test two CEB-Rs.)

But!  The WFC appears to be working, and better than it did before the mission!  First indications point to less than 4 electrons of RMS noise per pixel, vs. 4.5 to 5.5 before the failure.  So the new electronics actually works better than the old electronics. (Further testing may reveal other issues that can be alleviated at the cost of additional noise, so it’s not guaranteed yet, but it sure is looking good.)

And Oh By The Way, over the total of five EVA’s, the astronauts completed every task planned for the mission.  COSTAR was replaced by COS, WFPC2 was replaced by WFC3, two battery packs were replaced, all three gyroscope pairs were replaced, the Fine Guidance Sensor was also, and they replaced the outer insulating blankets on three areas.

A good week for NASA.  A good week for science.

2 Responses to “ACS Alive and Well”

  1. 1 digi421

    That’s great news, shame about the ACS HRC though. But for an almost 20 year old piece of equipment (in space), it’s ok have one issue.

  2. 2 Maggie

    That’s really great news!

Comments are currently closed.