The Great Motor Mystery

Abstract

In September and October 2010 MakerBot Industries received reports from the community about motors failing mid-build and some failing to rotate. Our motor supply was examined in response. The mid-print failing issue seems to have been caused by a bug in ReplicatorG. There is a hardware fix for seized motors.

Community reports

  • A working motor in an MK5 extruder read 50 Ohms across the terminals while a "busted" motor read 25 Ohms.
  • A MakerBot with a MK5 extruder, a heated platform, and the relays to go with them is able to start a build, but the extruder stops printing when the raft is finished. Sometimes it will get part-way though the first layer, or a build a few layers, then everything will stop. RepG will often give the "Read time out" message. Steps to reproduce error: After the extruder is heated up, starting and stopping the extruder motor a few times will cause the control panel to become unresponsive. After that happens, sometimes resetting the extruder board, will fix things. This has been tested with and without the filament in the extruder; things stop working after a few on/off cycles. Same response on a second motor. Both motors have circuit boards. Same results with extruder firmware v2.3 and v2.4 w/ unicorn.
  • A motor seizes up after a few minutes. The motor is removed form the extruder and gotten started. The motor typically seizes again.
  • User received a motor dead on arrival (DOA).

Quality testing

flickr:5079905443

Reports indicate motor quality as a potential failure point. A group of 137 MakerBot-50003 motors from Kysan were tested to quality. The motors do not carry serial numbers or lot numbers. These motors were taken from a single shipment.

Each motor was tested to observe if it can oscillate rapidly and do a full rotation in forward and reverse and its resistance measured. This test was performed under no load. A test rig was constructed. There are two extruder controllers powered by an ATX power supply. The extruder controllers are flashed with firmware that changes the direction of the motor connected to H-bridge A many times with a range of time intervals between each direction change (wobble test). Then the motors turns for 37.5 s clockwise (approximately 1.25 turns) then for 37.5 s counter-clockwise (rotation test). The controller repeats this procedure on H-bridge B. The extruder controller continues to repeat the test, alternating between H-bridge A and H-bridge B. Each motor's rotation and sound were observed. Flags made from tape and magnets were used to better observe the motor shaft's rotation. Piezoelectric buzzers are connected to digital pins D10 to indicate test phase changes.

During this test it was found that motors fresh from the box (never run) have a resistance upwards of 150 Ohms while motors after the stress test have a resistance of approximately 42 Ohms.

Motors are set aside if they sound significantly different from the other motors and are loud enough to hear. 5 motors were set aside for deviant sounds. 1 rotated much faster than the others. 1 died during the wobble test, failing the rotation test. There is not yet a correlation between motor sound and motor failure rates. In response to the DOA motor MakerBot immediately implemented the policy of testing every motor before it ships to customers. During this test it was found that motors fresh from the box (never run) have a resistance upwards of 150 Ohms while motors after the stress test have a resistance of approximately 42 Ohms.

Motor dissection

flickr:5080500364

Six motors were selected from the initial batch of suspicious motors. These were dissected in order to observe the interior and find a possible correlation between visual properties and motor failure. There appear to be two distinct forms of motors. Kysan must be shipping us two distinct motors. While it is normal industry practice to use slightly different parts as long as they meet the same specification, this could be where the problem is. If one type of motor has a significantly higher failure rate than the other, it may help us develop a solution.

Motor comparison

Here you can examine the two types of motors.

flickr:5079903585

The white plastic rear of the motor can be easily removed by unscrewing the two screws holding it in place. Their exteriors appear the same. The interiors have noticeable differences. Version A is on the left, version B on the right.

flickr:5080498432

Each motor's rear will be in one of two configurations. The circuit board appears to be for electrical noise suppression. Version A is on the left, version B on the right.

flickr:5080499056

These are the commutators. Version A is on the left, version B on the right.

flickr:5082455792
flickr:5079904075

These are the casings for the motor housings. Notice the shininess difference. Version A is on the left, version B on the right.

flickr:5079903355

The interior of the motor housing appears to have little difference between them except for the metal texture due to the machining. The magnets in version A appear to be weaker in our test group. Version A is on the left, version B on the right.

flickr:5080499526

This is the gear head housing. Notice the difference in hole arrangement. Version A is on the left, version B on the right.

flickr:5080499802

These are the coils. One looks hand-wound while the other appears to be machine wound. Version A is on the left, version B on the right.

flickr:5079904877

This is the top of the gear head housing. The gear you see is connected directly to the motor shaft. Notice the difference in the cut outs. Version A is on the left, version B on the right.

No noticeable difference was observed in the function of the two types of motors.

Motor survey

Since we only tested motors from a single batch a survey was taken of the community. If enough data is gathered, we may be able to find a correlation between motor failure rate and some other property. This would help us develop a solution. This survey continued while research continued.

User feedback shows a strong correlation a motor having a circuit board and having a shiny ring. After further research there appears to be no correlation between either type of motor and motor failure.

Electrical Characteristics

Possible subtle electrical differences between the two types of motors may be causing electrical interference and data packet loss.

To measure the current through the motor we used a ¼ Ohm resistor connected to the H-bridge sense pin and the Rigol DS1052E oscilloscope. To load the motor a pair of vice grip pliers were clamped to the motor shaft which were held by hand to torque the motor shaft to near stalling.

Bare motor examination

In this experiment a non-boarded motor (without gear box) was examined using an oscilloscope. The image below are were taken while using non-zero reversal parameters. Reversal parameters: time to pause = 5 ms, time to reverse = 500 ms, time to advance = 300 ms, min. extrusion time before reversal = 300 ms.

In the following oscilloscope images each horizontal block represents a 20.00 ms time interval. Each vertical block represents 50 mV of electric potential. Given the ¼ Ohm resistor connected to the H-bridge sense pin, each vertical block also represents 200 mA of current.

flickr:5080502442

While starting a motor with no load the initial spike reaches approximately 125 mV, 500 mA and settles at approximately 25 mV, 100 mA.

flickr:5079906321

When starting there is not significant difference in the initial spike. However, the motor settles at approximately 50 mV, 200 mA.

flickr:5079906785

Stopping an unloaded motor we observe an initial spike of approximately 250 mV, 1000 mA. In this image we see the motor moving forward for the first 40 ms. The next 5 ms is the pause time where the voltage drops to zero. The large spike upwards is from the motor reversing. Remember, we are looking at the absolute value of the voltage, so we are seeing a positive voltage when the actual voltage here is negative.

flickr:5080501178

A stopping motor experiences an initial spike of 200 mV, 800 mA.

Spike comparison

In this experiment the initial voltage and current spike of five boarded and five non-boarded motors were recorded. The machine is reset after each trial. The machine is powered down when switching motors. Red motor wire connected to 1A, black to 1B. Motherboard firmware v2.3, extruder firmware v2.5, PWM=255, no reversal parameters. Motor are placed on their sides. All motors passed the stress test and rotation tests.

Motor Trial 1 Trial 2 Trial 3 Trial 4 Trial 5 Average Deviation
B1 84 82 82 74 80 80.4 2.72
B2 68 72 74 84 84 76.4 6.08
B3 82 82 82 82 82 82 0
B4 80 80 82 84 88 82.8 2.56
B5 72 76 76 78 76 75.6 1.44
Average 79.44 ± 2.56
Voltage spike (mV). Unloaded, dull-finished motor stopping from forward.
Motor Trial 1 Trial 2 Trial 3 Trial 4 Trial 5 Average Deviation
A1 78 80 80 80 74 78.4 1.92
A2 86 102 96 78 80 88.4 8.84
A3 86 86 80 86 86 84.8 1.92
A4 78 76 76 80 78 77.6 1.28
A5 80 72 74 76 78 76 2.4
Average 81.04 ± 3.272
Voltage spike (mV). Unloaded, shiny-finished motor stopping from forward.

Performing this test several times on motor A1 there appeared to be no significant difference in the initial spike between no PWMing and PWM=250. This supports the idea that PWMing and starting/stopping the motor are electrically identical.

From this data set we can conclude that in our current stock of motors there is no difference in the voltage spike between the two types of motors. Further evidence that the motors are functionally identical.

Repeatable failure

PWMing is believed to give sketchy results. PWMing is just turning the motor on and off. Perhaps turning the motor on and off gives the same bad result as PWMing. We turn the motor on and off during the reversal parameters.

The reversal parameters were written in gcode and repeated many time. When executed the motor motor sticks in a forward, reverse, or halted state at approximately the 19th cycle. This happens nearly every time. I tested five shiny and five ringed motors on various extruder controllers using firmware v2.5 and v1.4.

M108 S255 (set extruder speed to maximum)

M101 (Extruder on, forward)
G04 P300 (Wait t/1000 seconds)
M103 (Extruder off)
G04 P5 (Wait t/1000 seconds)
M102 (Extruder on, reverse)
G04 P500 (Wait t/1000 seconds)
M101 (Extruder on, forward)
G04 P300 (Wait t/1000 seconds)
M103 (Extruder off)
G04 P5 (Wait t/1000 seconds)
M102 (Extruder on, reverse)
G04 P500 (Wait t/1000 seconds)
...
M101 (Extruder on, forward)
G04 P300 (Wait t/1000 seconds)
M103 (Extruder off)
G04 P5 (Wait t/1000 seconds)
M102 (Extruder on, reverse)
G04 P500 (Wait t/1000 seconds)

Is the H-bridge latching?

flickr:5094928680

H-bridge latching would be indicated by a motor constantly rotating in one direction in the presence of a blinking LED. A blinking LED indicates the alternating voltages to the H-bridge are behaving as defined in the gcode. The constant motor motion indicates the H-bridge is latched and not changing output based on its input.

The LEDs and motor are visually observed while the Rigol oscilloscope is connected to the H-bridge GND and OUTA pins. This image shows the cycles of a short 300 ms forward rotation followed by a 500 ms reverse rotation and the proceeding failure where the motor sticks in the reverse rotation state.

Results of this test: The motor is observed to rotate in one direction. The H-bridge LEDs are not blinking. Conclusion: the H-bridge is not likely latching.

Is the H-bridge getting logic input?

flickr:5094332123

Logic signals input to the H-bridge determine its behavior. If we see output behavior inconsistent with the logic input, we know there must be something wrong with the H-bridge internals. If we detect logic input inconsistent with what we expect as output from the ATmega chip, the H-bridge is probably fine.

flickr:5094414531

In this image the oscilloscope is connected to H-bridge GND and Phase pins. The Rigol monitored the Phase pin while the motor and LEDs were visually observed. This image shows the cycles of one 300 ms forward rotation followed by a 500 ms reverse rotation. The plateau is when the motor sticks in the reverse rotation state. From this we can conclude the ATmega chip is not outputting the desired phase logic.

Further testing show the Sleep pin in a constant 5 V state before and after the fail, as expected. The chip is in sleep mode when the Sleep pin is in the low logic state. I think this is connected directly to the 5 V line.

The Enable pin is at 5 V before the fail except for the short 5 ms pauses. This is expected. After the fail the Enable pin remains constant at 5 V.

The Mode pin and GND are always at 0 V. When I connect Mode and V_BB to the oscilloscope the board resets. What's going on here?

All this points to some problem at the microcontroller. It's probably not the firmware. The motor problem is found with firmware v1.4 and v2.5, between which there was pretty much a complete code rewrite. It must be on the board.

Is the microcontroller outputting the logic we expect?

flickr:5095238446

The oscilloscope was connected to the D7 (pin 12) of the ATmega chip. The logic is output until the motor fail is observed. The problem appears to be within the microcontroller or something causing the microcontroller to puke.

Is power to the microcontroller dipping and causing a reset?

The oscilloscope is connected to GND (pin3) and V_CC (pin 6) of the ATmega. The power remains constant. conclusion, the chip is not losing power.

The crystal

flickr:5094896757

The oscilloscope is connected to ATmega GND (pin 3) and a pin on the 16 MHz oscillator. When connected to one leg the extruder controller board cannot communicate with the mother board. Using the other pin and a motor speed of PWM=255 gives this wave form.

flickr:5095227305

With PWM=255 the motor rotates forward with with the oscillator giving the expected wave form. When the motor is stopped the oscillator gives the wave form pictured here. That nasty bit in the middle happens only when the motor is stopped. Stopping from forward or reverse gives similar results. Starting the motor does not mess up the wave form.

flickr:5094896211

Using PWM is just starting and stopping the motor we can expect that we would see a wave form similar to that of the motor stopping. Even with PWM=254 we get the same result in this image.

Using the internal oscillator

8 MHz. The firmware was changed to keep the baud rate the same since it normally runs on 16 MHz.

H-bridge comparison

flickr:5099756452
flickr:5099156583

Due to differences in routing on the extruder controller PCB it is believed that the H-bridge connected to 2A/2B will be more reliable than 1A/1B. Using power supply PS1, motor B1, and extruder controller EC1 (labels written on the objects) the average time to failure will be determined for each H-bridge. Images of the experimental set up are included. Sometimes the error is not reproducible.

1A/1B 2A/2B
16.4 16.2
16 15.3
16 15.8
16.2 15.9
15.5 16
15.4 15.8
15.4 15.4
15 15.3
16 16.1
15.4 15.3
15.4 15.6
average = 15.7 s average = 15.7 s
std dev = 1.75 s std dev = 0.29 s
Average time to failure on each H-bridge

While collecting this data a discovery was made about the particular circumstances under which the motor will fail. The motor fails at approximately 16 seconds (~19 cycles) or not at all. This was believed to be a random phenomenon, but is now determined to be dependent on the state of the machine. When running the test gcode on a freshly reset machine the motor will not fail. If, however, during a build Stop is pressed and the code run again, the motor will fail. This is at least true for both H-bridges. Failure does not happen on the second run if the board is freshly reset, program run, allowed to complete, and run again. Bug filed.

Time-based or cycle-based failure?

Let's see if the failure is dependent on time or on the number of cycles. If the failure is time-based, the motor should fail after about 16 seconds independent of the cycle time. If the failure is cycle-based, the motor should fail after about 19 cycles independent of the cycle time.

The cycle time is modified by replacing all instances of "G04 P300 (Wait t/1000 seconds)" with "G04 P900 (Wait t/1000 seconds)". This results in the extruder rotating forward for 900 ms rather than 300 ms. Our original cycle time was 805 ms and is now 1405 ms. Our new cycle time is 1.75 times our old. A time-based failure should appear at approximately 15.7 seconds. A cycle-based failure should appear at approximately (1.75)(15.7 s)=27.5 s.

M108 S255 (set extruder speed to maximum)

M101 (Extruder on, forward)
G04 P900 (Wait t/1000 seconds)
M103 (Extruder off)
G04 P5 (Wait t/1000 seconds)
M102 (Extruder on, reverse)
G04 P500 (Wait t/1000 seconds)
M101 (Extruder on, forward)
G04 P900 (Wait t/1000 seconds)
M103 (Extruder off)
G04 P5 (Wait t/1000 seconds)
M102 (Extruder on, reverse)
G04 P500 (Wait t/1000 seconds)
...
M101 (Extruder on, forward)
G04 P900 (Wait t/1000 seconds)
M103 (Extruder off)
G04 P5 (Wait t/1000 seconds)
M102 (Extruder on, reverse)
G04 P500 (Wait t/1000 seconds)

Running the test five times on H-bridge 2A/2B gives the following data.

2A/2B
27.4
25.8
25.6
25.8
27.8
average = 26.5 s
std dev = 0.9 s

The new average failure time of 26.5 +/- 0.9 s pretty close to our predicted value of 27.5 seconds. From this we can conclude that the failure is caused by the number of cycles.

So what's so important about the number of cycles? Perhaps the buffer size has something to do with this. It may be that the extruder controller reads to the end of the buffer and stops. Hob large is the buffer? How large are the packets? How many packets are required to execute those 19 cycles? This may point to a software solution here. It still looks like there is some sort of interference that is causing the extruder controller to to bazonkers.

Looking at the oscilloscope the packets are a little over 1 ms apart when using the normal gcode test.

The buffer is on the motherboard, not the extruder controller.

Always check your cables

flickr:5101821165

It looks like one of the oscilloscope cables is bad and has been introducing noise. The one probe's ground lead is broken. It didn't take much force to pull it apart. With the remaining good probe I confirmed that there is still noise on the oscillator line when the motor PWMing or doing start/stop cycles.

Does the problem exist for different combination of Gen3 and Gen4 motherboard and extruder controllers?

Yes. A Gen4 extruder controller was tested with the same Gen3 motherboard as before. The problem remains reproducible. The problem must exist on the motherboard.

Is it a firmware issue?

Use a Gen3 motherboard with firmware v1.6 and extruder controller with firmware 2.5. The problem is not exactly reproducible. The motor seems to fail randomly rather than at 15.7 seconds.

EC Firmware MB Firmware Fail time
v2.5r0 v2.3r0 15.7 s
v2.5r0 v2.2.r0 8.6, random after
v2.5r0 v2.1r1 8.5, random after
v2.5r0 v1.6 random

This indicates the issue may be somewhere in the changes made between motherboard firmware v1.6 and v2.1. Bus retries were added in the 2x series. Bus retries were increased in v2.3.

Does the motor fail into one rotation state?

Is the motor rotating in the same direction each time it fails? Do a number of fails and see what the state distribution is should be a 3:5 ratio. G3 boards MB firmware v2.3r0, EC v2.5r0. Results: 16 tries, 15 failed rotating in reverse, 1 appeared to not fail at 15.7 seconds, but failed around 28 seconds rotating in the forward direction.

ReplicatorG bug FIXED

There is a bug in ReplicatorG 20 and below which causes many prints to fail if they are run from ReplicatorG after the Stop button is pressed. The fix will be released in ReplicatorG 21. As a workaround until the ReplicatorG 21 release always press the Reset button in ReplicatorG after you press the Stop button.

Seizing motors FIXED

In some motors the gears have been binding because the first gear in the gear box is too close to the gear on the motor shaft. A seized motor can be fixed by adjusting the gearbox relative to the motor housing to increase the space between the first two gears. This can be done by

  1. Remove the front of the gear box
  2. Loosen the screws at the bottom of the gearbox
  3. Move the gear box by hand or with a mallet
  4. Check motor function by connecting directly to 12 V on the power supply (yellow and black wires).
  5. Tighten the screws again
  6. Reattach the front of the gear box.

Revamp your Extruder Board! - FIXED

If you've been hit with communication problems from your motor or a dying H-Bridge, check out this simple but robust H-bridge fix which an intrepid user devised.

Keep Your Motor Running Using A Relay Board Kit - FIXED

For those experiencing a motor that stops in the middle of prints very early in the component's life may wish to experiment with using a Relay Board kit between the Extruder Controller DC drive ports and the motor so that the primary power for the motor is drawn directly from the PSU instead of via the EC. Relay Board kits can be obtained from the store.makerbot.com and assembled as per the instructions.

Once you have this solution in place, you need to clamp your pwm options for your bot to 255-only to prevent burning out your relay board. Here's how to do it.

1. Copy this file into replicatorg\machines\ (Right-click and "Save Linked File As…" or similar options.)
2. Start RepG
3. Select the appropriate machine driver (Machine->Driver):
for automated build platform: 'Thingomatic w/ ABP and relay-driven MK5'
for heated build platform: 'Thingomatic w/ HBP and relay-driven MK5'
4. Run as normal. Note that the status window will show a warning
whenever a machine speed is overridden.

Unless otherwise stated, the content of this page is licensed under GNU Free Documentation License.