Filter
Top Products
Appendices
Specifications
A
A-11
Resistance Measurements
Resolution
See Table A-13.
Table A-13. Resistance Measurements - Resolution.
Range Resolution Typical Full
Scale Voltage
Maximum
Current
Through
Unknown
Maximum
Open Circuit
Voltage
Slow Fast
300Ω 10 mΩ 0.1Ω 0.22V 1 mA 3.2V
3 kΩ 0.1Ω 1Ω 0.25V 110 µA 1.5V
30 kΩ 1Ω 10 Ω 0.29V 13 µA 1.5V
300 kΩ 10 Ω 100 Ω 0.68V 3.2 µA 3.2V
3 MΩ 100 Ω 1 kΩ 2.25V 3.2 µA 3.2V
10 MΩ 1 kΩ 10 kΩ 2.72V 3.2 µA 3.2V
Accuracy
See Table A-14.
Table A-14. Resistance Measurements - Accuracy (Four-Wire)
4-Wire Accuracy ±(%±Ω)
Range
18°C to 28°C
0°C to 60°C
90 Days, Slow 1 Year, Slow 1 Year, Fast 1 Year, Slow 1 Year, Fast
300Ω 0.013% + 20 mΩ 0.014% + 20 mΩ 0.014% + 0.2Ω 0.031% + 20 mΩ 0.031% + 0.2Ω
3 kΩ 0.015% + 0.2Ω 0.016% + 0.2Ω 0.016% + 2Ω 0.039% + 0.2Ω 0.039% + 2Ω
30 kΩ 0.013% + 2Ω 0.014% + 2Ω 0.014% + 20Ω 0.039% + 2Ω 0.039% + 20Ω
300 kΩ 0.020% + 20Ω 0.021% + 20Ω 0.021% + 200Ω 0.050% + 20Ω 0.050% + 200Ω
3 MΩ 0.059% + 200Ω 0.063% + 200Ω 0.063% + 2 kΩ 0.231% + 200Ω 0.231% + 2 kΩ
10 MΩ 0.168% + 2 kΩ 0.169% + 2 kΩ 0.709% + 20 kΩ 0.573% + 2 kΩ 0.923% + 20 kΩ