Erratic behavior, possibly contact related


M. Collins
 

  I'm part of a group at Lowell Observatory tasked with acquiring a new 1 meter-class telescope and instrumentation to replace one of our older research telescopes which is no longer viable to maintain. One of the associated activities is an assessment of sites available for the new telescope. For that, we have some observers using 14" SCTs with a mask that allows light to pass directly through a ca. 80 mm aperture on one side of the secondary, with a second aperture opposite that has an optical wedge in place to divert the light passing through slightly. At the focal plane, a double image is produced. Each image is affected by seeing effects in the atmosphere, resulting in small differential movements of the images with respect to each other. By measuring the amplitude of these movements as a function of time, a quantitative measure of seeing quality can be obtained. For our tests, two telescopes are operated simultaneously at candidate sites, each recording sequences of images of second magnitude and brighter stars. The data sets collected are later processed using analysis software and the results compared to determine which site provides better seeing on average.

  One of the mounts was the original Celestron fork driven using a synchronous motor and variable rate drive controller. It's not too bad so far as it goes, however there is sufficient periodic error to make sub-frame imaging difficult. Also, the mount does not lend itself to remote operation. I have a 1200GTO mount which has been in storage for a couple of years awaiting a new home, so I volunteered it for the duration of the seeing tests. It was a simple matter to fabricate an adapter plate to go between the mount and the C14 tripod, and we were ready to go on sky last Monday. The first night out, we polar aligned and had just started capturing data sets (1000 frames at 20 ms cadence, repeated at 10 minute intervals) with Deneb as a target when clouds rolled in. The mount has a CP3 controller I recently purchased on the used equipment market. Since that controller can operate at up to 16 V, a 15 V supply with the appropriate locking connector was secured to the tripod. Everything was in good working order up the point that I departed the site. The observer stayed on and was able to capture a few more sets later that night before the clouds eventually prevailed.

  The following night was clear, and it was during initial start-up activities in twilight that things started going wrong. The report I received afterward was that the mount ran away in declination while seeking to the first target, slewing far beyond and not responding to the stop button on the hand controller. The observer powered off, manually moved to Park 1, and powered on again. Before it was possible to perform a sync, it ran away again, so the observer shut down for the night.

  I visited the site the following day, bringing with me the original CP2 controller as well as the 12 V power supply which I had used when the mount was in my observatory a few years ago. Without changing any components but after confirming that all of the connections were secure, I found that the mount was again running away, sometimes in right ascension as well as declination. Replacing the power supply had no effect, so I also substituted the CP2. Again, the problem resurfaced within a few seconds of the first attempt to move in either axis. I also noted that the motors would stop and start erratically during slews, sometimes in very rapid succession, regardless of the power supply or controller being used.

  The only remaining electronic element was the hand controller. I removed the cover and first measured the voltage of the lithium cell. No problem there, as it was about 3.1 V. I next confirmed continuity of the four wires which are used in the coiled cable, aggressively flexing the cable at each end in case an internal conductor is intermittent at certain orientations. Still no sign of a failure. The last step was simply cycling (disconnecting then reconnecting) each of the cable connectors which mate with the main board in the controller. Since these connectors used tinned contacts, it's possible for marginally- to non-conducting oxide layers to form over time. Cycling the connectors will generally abrade the oxide sufficiently to expose new metal-to-metal contact, renewing the connection (for a while).

  After reassembling the hand controller and going through the power up sequence again, the mount returned to its usual, flawless self. Seeing no issues after ten minutes or so, I reinstalled the CP3 and 15 V power supply. I drove it around the daytime sky for at least an hour, initially with the hand controller, then using Stellarium across a serial link, and never saw the slightest indication of an error. It's been in use without issue from twilight until after midnight each night since.

  I don't know details about the communications across the serial link between the hand controller and the CPx, so it's not possible to propose a failure mechanism related to faulty or intermittent contact that can explain the behavior we saw. That said, cycling the connectors in the hand controller was the only action taken which appears to have corrected the problem. To my knowledge, that controller was purchased along with the mount, probably sometime around 1990, so an age related contact issue is not beyond the realm of plausibility. Going forward, it will be my practice to slide each of the connectors off and back onto the pins inside the controller once or twice any time I open it up to replace the coin cell.

  Passing the information along in case it might be useful to someone else. If what I did solved what appeared to be a serious problem, it's a very easy fix.


Roland Christen
 

Your problem has nothing to do with the hand controller. It is just a planetarium program that only sends RA/Dec co-ordinates to the mount when you enter it. Otherwise it is silent and does not communicate with the mount for any reason.

The symptoms that you are experiencing suggest that the Dec motor encoder is not sending signals to the CP3 (or CP2) due to a faulty connection on the Dec portion of the Y cable. You may have to take the cover off the end of the cable connector and see if the wires are ok on the pins.

Here is one test you can do to see if it's the Dec connection: Unplug the Dec cable, plug the RA cable into the RA motor and turn on the mount. Does the RA track? Can you move the RA with the keypad E-W buttons at 12x, 64x, 600x? If so, then the RA portion is ok.

Now take the RA cable off the RA motor and plug it into the Dec motor. turn on the mount. If the Dec works the same as RA then the Dec motor and encoder are working, and the fault would be in the cable. If the Dec runs away, then there is something wrong with the Dec motor encoder.

Rolando

-----Original Message-----
From: M. Collins <aegle_observatory@...>
To: main@ap-gto.groups.io
Sent: Wed, Sep 15, 2021 6:26 pm
Subject: [ap-gto] Erratic behavior, possibly contact related

  I'm part of a group at Lowell Observatory tasked with acquiring a new 1 meter-class telescope and instrumentation to replace one of our older research telescopes which is no longer viable to maintain. One of the associated activities is an assessment of sites available for the new telescope. For that, we have some observers using 14" SCTs with a mask that allows light to pass directly through a ca. 80 mm aperture on one side of the secondary, with a second aperture opposite that has an optical wedge in place to divert the light passing through slightly. At the focal plane, a double image is produced. Each image is affected by seeing effects in the atmosphere, resulting in small differential movements of the images with respect to each other. By measuring the amplitude of these movements as a function of time, a quantitative measure of seeing quality can be obtained. For our tests, two telescopes are operated simultaneously at candidate sites, each recording sequences of images of second magnitude and brighter stars. The data sets collected are later processed using analysis software and the results compared to determine which site provides better seeing on average.

  One of the mounts was the original Celestron fork driven using a synchronous motor and variable rate drive controller. It's not too bad so far as it goes, however there is sufficient periodic error to make sub-frame imaging difficult. Also, the mount does not lend itself to remote operation. I have a 1200GTO mount which has been in storage for a couple of years awaiting a new home, so I volunteered it for the duration of the seeing tests. It was a simple matter to fabricate an adapter plate to go between the mount and the C14 tripod, and we were ready to go on sky last Monday. The first night out, we polar aligned and had just started capturing data sets (1000 frames at 20 ms cadence, repeated at 10 minute intervals) with Deneb as a target when clouds rolled in. The mount has a CP3 controller I recently purchased on the used equipment market. Since that controller can operate at up to 16 V, a 15 V supply with the appropriate locking connector was secured to the tripod. Everything was in good working order up the point that I departed the site. The observer stayed on and was able to capture a few more sets later that night before the clouds eventually prevailed.

  The following night was clear, and it was during initial start-up activities in twilight that things started going wrong. The report I received afterward was that the mount ran away in declination while seeking to the first target, slewing far beyond and not responding to the stop button on the hand controller. The observer powered off, manually moved to Park 1, and powered on again. Before it was possible to perform a sync, it ran away again, so the observer shut down for the night.

  I visited the site the following day, bringing with me the original CP2 controller as well as the 12 V power supply which I had used when the mount was in my observatory a few years ago. Without changing any components but after confirming that all of the connections were secure, I found that the mount was again running away, sometimes in right ascension as well as declination. Replacing the power supply had no effect, so I also substituted the CP2. Again, the problem resurfaced within a few seconds of the first attempt to move in either axis. I also noted that the motors would stop and start erratically during slews, sometimes in very rapid succession, regardless of the power supply or controller being used.

  The only remaining electronic element was the hand controller. I removed the cover and first measured the voltage of the lithium cell. No problem there, as it was about 3.1 V. I next confirmed continuity of the four wires which are used in the coiled cable, aggressively flexing the cable at each end in case an internal conductor is intermittent at certain orientations. Still no sign of a failure. The last step was simply cycling (disconnecting then reconnecting) each of the cable connectors which mate with the main board in the controller. Since these connectors used tinned contacts, it's possible for marginally- to non-conducting oxide layers to form over time. Cycling the connectors will generally abrade the oxide sufficiently to expose new metal-to-metal contact, renewing the connection (for a while).

  After reassembling the hand controller and going through the power up sequence again, the mount returned to its usual, flawless self. Seeing no issues after ten minutes or so, I reinstalled the CP3 and 15 V power supply. I drove it around the daytime sky for at least an hour, initially with the hand controller, then using Stellarium across a serial link, and never saw the slightest indication of an error. It's been in use without issue from twilight until after midnight each night since.

  I don't know details about the communications across the serial link between the hand controller and the CPx, so it's not possible to propose a failure mechanism related to faulty or intermittent contact that can explain the behavior we saw. That said, cycling the connectors in the hand controller was the only action taken which appears to have corrected the problem. To my knowledge, that controller was purchased along with the mount, probably sometime around 1990, so an age related contact issue is not beyond the realm of plausibility. Going forward, it will be my practice to slide each of the connectors off and back onto the pins inside the controller once or twice any time I open it up to replace the coin cell.

  Passing the information along in case it might be useful to someone else. If what I did solved what appeared to be a serious problem, it's a very easy fix.

--
Roland Christen
Astro-Physics


M. Collins
 

  As I noted, the mount has operated without any errors over many hours in the past week, so at the moment there is nothing to investigate. Prior to exchanging any of the electronic components, I did check to see that all of the connections were sound, including the three associated with the Y cable. While I didn't remove the shells to inspect the soldered connections to the pins on the three connectors, the cable shows no sign of damage and it seems likely that an intermittent contact between the motor controller and motors would have resurfaced by now. So at best, there was an fault somewhere which had cleared coincidentally when the hand controller was reconnected.

  Also, the RA axis exhibited the same erratic behavior as reported for the declination axis when I was initially investigating last Wednesday, running away or rapidly stopping/starting during a slew. Whatever was causing problems appears to have been common to both axes. Perhaps a cable fault could cause such a thing. No other single element (other than the hand controller, which you've ruled out) was in place each time failures occurred. (And to be clear, only the hand controller, Y cable and power were connected to either the CP2 or CP3 during the initial investigation. The USB/serial translator was not connected until after the mount had operated using the hand controller for more than half an hour after the problems had disappeared.)

  If we see further issues, I'll start by checking the connections through the Y cable.


Roland Christen
 


So at best, there was an fault somewhere which had cleared coincidentally when the hand controller was reconnected.
You probably had corrosion on the pins of the Y cable and possibly on the motor box as well. Inserting the cable a couple of times might have wiped the pins somewhat clean, but there might still be a poor contact. You might want to clean all the pins of the cable and mating connectors with a contact cleaner and perhaps add some anti-corrosion lube to them to prevent problems in the future. Pins can deteriorate over time, especially if they are subjected to high humidity.

In these mounts, runaways are almost always caused by poor connection between the motor encoder and the CP servo controller. Reason is that there is very low level signal with almost no current and fairly low voltage being run on these signal lines. Therefore if there is no current to speak of, the corrosion acts as a fairly good insulator. By contrast, the two power connections to the motors carry a fair amount of current, so the corrosion is easily bridged by the voltage present on those pins.

Runaways occur when the encoder signals don't reach the servo controller, so the controller thinks that the motor is not turning and supplies more and more voltage and current to turn the motor shaft. Of course, the motor shaft is turning, but without the encoder feedback, the controller runs open loop.

Rolando


-----Original Message-----
From: M. Collins <aegle_observatory@...>
To: main@ap-gto.groups.io
Sent: Thu, Sep 16, 2021 12:36 am
Subject: Re: [ap-gto] Erratic behavior, possibly contact related

  As I noted, the mount has operated without any errors over many hours in the past week, so at the moment there is nothing to investigate. Prior to exchanging any of the electronic components, I did check to see that all of the connections were sound, including the three associated with the Y cable. While I didn't remove the shells to inspect the soldered connections to the pins on the three connectors, the cable shows no sign of damage and it seems likely that an intermittent contact between the motor controller and motors would have resurfaced by now. So at best, there was an fault somewhere which had cleared coincidentally when the hand controller was reconnected.

  Also, the RA axis exhibited the same erratic behavior as reported for the declination axis when I was initially investigating last Wednesday, running away or rapidly stopping/starting during a slew. Whatever was causing problems appears to have been common to both axes. Perhaps a cable fault could cause such a thing. No other single element (other than the hand controller, which you've ruled out) was in place each time failures occurred. (And to be clear, only the hand controller, Y cable and power were connected to either the CP2 or CP3 during the initial investigation. The USB/serial translator was not connected until after the mount had operated using the hand controller for more than half an hour after the problems had disappeared.)

  If we see further issues, I'll start by checking the connections through the Y cable.

--
Roland Christen
Astro-Physics


M. Collins
 

On Thu, Sep 16, 2021 at 09:43 AM, Roland Christen wrote:
You probably had corrosion on the pins of the Y cable and possibly on the motor box as well.
  This seems to be the most plausible explanation. It's odd that everything worked during bench testing and the first night at the site, then the problem surfaced and persisted after cycling the connectors, replacing the mount controller, etc., and finally went away without any actions which involved the Y cable connections. But I've seen enough strange failure modes involving old equipment and poor contacts to know that almost anything is possible.

  Are there any common connections through the Y cable, e.g., a signal ground pin which is connected to both motor assemblies? Poor contact at one pin which can affect both motor drives would fully explain the symptoms observed last week. (The mount is at one of our remote sites, so I can't readily pull the cable to see how it's wired.)
Runaways occur when the encoder signals don't reach the servo controller, so the controller thinks that the motor is not turning and supplies more and more voltage and current to turn the motor shaft. Of course, the motor shaft is turning, but without the encoder feedback, the controller runs open loop.
  A question about this. On more than one occasion when one or both axes were running away, pressing the stop button on the hand controller had no effect and it was only possible to stop the motors by pulling the power connection. Does the fact that the motors continued to be powered despite the stop operation point to another possible fault?

  Thanks for the details regarding the servo loops and hand controller interface.


Roland Christen
 


Does the fact that the motors continued to be powered despite the stop operation point to another possible fault?
Stop is a software command, so it requires a working servo system. Once the encoder connection fails, the servo loop is broken and no software commands will do anything. In later versions of the servo software there is a motor fail limit which will stop power to the motors after approx 2 seconds. You may have an older version of the software in your CP3.

Roland


-----Original Message-----
From: M. Collins <aegle_observatory@...>
To: main@ap-gto.groups.io
Sent: Thu, Sep 16, 2021 1:33 pm
Subject: Re: [ap-gto] Erratic behavior, possibly contact related

On Thu, Sep 16, 2021 at 09:43 AM, Roland Christen wrote:
You probably had corrosion on the pins of the Y cable and possibly on the motor box as well.
  This seems to be the most plausible explanation. It's odd that everything worked during bench testing and the first night at the site, then the problem surfaced and persisted after cycling the connectors, replacing the mount controller, etc., and finally went away without any actions which involved the Y cable connections. But I've seen enough strange failure modes involving old equipment and poor contacts to know that almost anything is possible.

  Are there any common connections through the Y cable, e.g., a signal ground pin which is connected to both motor assemblies? Poor contact at one pin which can affect both motor drives would fully explain the symptoms observed last week. (The mount is at one of our remote sites, so I can't readily pull the cable to see how it's wired.)
Runaways occur when the encoder signals don't reach the servo controller, so the controller thinks that the motor is not turning and supplies more and more voltage and current to turn the motor shaft. Of course, the motor shaft is turning, but without the encoder feedback, the controller runs open loop.
  A question about this. On more than one occasion when one or both axes were running away, pressing the stop button on the hand controller had no effect and it was only possible to stop the motors by pulling the power connection. Does the fact that the motors continued to be powered despite the stop operation point to another possible fault?

  Thanks for the details regarding the servo loops and hand controller interface.

--
Roland Christen
Astro-Physics