Testing on HP Business Desktop PCs
Introduction......................................................................................................................................... 2
Importance of Desktop Testing............................................................................................................... 2
Test Tool Development.......................................................................................................................... 3
Hardware Evaluation Test Development .............................................................................................. 3
Software Image Development ............................................................................................................ 3
Types of Testing................................................................................................................................... 4
System Board Validation Testing ........................................................................................................ 4
Platform Validation Testing ................................................................................................................ 4
Mechanical Validation Testing........................................................................................................... 5
Module Functional Testing................................................................................................................. 5
System Integration Testing ................................................................................................................. 6
Performance Testing ......................................................................................................................... 7
Desktop Certifications....................................................................................................................... 7
Tested Configurations........................................................................................................................... 8
Hours of Testing................................................................................................................................... 8
Observations during Testing.................................................................................................................. 9
On-going Qualifications ....................................................................................................................... 9
For More Information ......................................................................................................................... 10
Introduction
Testing is an important part of the value supplied by HP to our customers. Many companies can
assemble personal computers. HP has the engineering expertise and resources to perform the tens of
thousands of hours testing every desktop platform we develop. This testing provides our customers
with peace of mind, knowing that hundreds of different tests have been performed to provide a stable
and reliable experience.
In our quest to continuously improve the customer experience, HP uses a disciplined and methodical
development and test process. We verify that each new desktop platform meets or exceeds HP quality
standards for structural integrity, reliability, and compatibility. This white paper describes the various
forms of testing performed by HP engineering when developing a new HP Business Desktop PC
platform.
Importance of Desktop Testing
Business Desktop PCs represent a large portion of the worldwide computer shipments and are often
the workhorses of business. As such, it is critical that they be durable and able to support a vast array
of operating systems, languages, applications, and add-in options. Helping to ensure this
compatibility and flexibility requires extensive, well-engineered testing.
In this industry, quality is a differentiator. Business customers recognize the value of a more reliable
PC, and that is a major factor in their purchase selection. The time and resources we invest in our
testing yield benefits to them and to HP. A well-tested product can lead to a more efficient factory,
lower costs, fewer failures in the field, and more satisfied customers.
The PC business is a dynamic industry in which new technologies are created and adopted. It is
imperative that we have a robust test process that can react to new technologies, verify their quality,
integrate them into our product lines, and get them to market quickly.
The following sections provide more details of the HP test tool development and capability and the
types of testing performed. These sections also provide a closer look at the investment of time in our
test process and how we track the results of tests conducted on HP Business Desktop PCs.
2
Test Tool Development
An essential element of the HP test process is our attention and dedication to continuously improving
the tests and processes used during product development. Two such areas of concentration are
Hardware Evaluation Test Development and Software Image Development. Hardware Evaluation Test
Development designs tests to help ensure that hardware components conform to published functional
engineering design specifications. Software Image Development architects image databases,
processes, and tools used to test and deliver software images on business desktop products.
Hardware Evaluation Test Development
The Evaluation Test Development (ETD) team designs and provides hardware and software test tools
necessary to help ensure that the hardware components of HP products conform to published
functional engineering design specifications. This engineering organization, established in 1988, is
shared by most of HP. The ETD tools are used throughout all R&D organizations within the Personal
Systems Group (PSG) and most R&D organizations within the Enterprise Systems Group (ESG), in
manufacturing sites, and by HP field engineers. The ETD tests enable product development teams to
better evaluate new designs and resolve most design issues, and they help improve manufacturing
volume production.
• Hardware and Software design evaluation tools—ETD tests emulate most states of a targeted sub-
system, they are low cost and easy to setup, and they enable product teams to test a large number
of units, which helps to identify yield failures. These tools provide detailed error messages that aid
in debugging, and they provide reproducible test cases.
• ETD HW and SW tests are executed on all business desktop products and are used on the
following product sub-systems:
– Microprocessors and chipsets, including cache memory, interrupts, and timers
– System memory
– PCI/PCI-X/AGP/PCI Express Graphics subsystems
– Hard Drive and Removable Storage interfaces
– Network Interface Cards and modems
– Parallel, Serial, and USB 1.0/2.0 ports, Bluetooth, and Firewire
– Power Management
– Cardbus/PCMCIA
Software Image Development
Software Image Development is optimized for time-to-market delivery, quality, and release integrity. A
dedicated tool development team was formed in 1998. Their charter was to design software image
databases, processes, and tools that encompass the following elements:
• Test Management—Database tools enable test teams to generate and manage test plans and test
procedures in a centralized, repeatable environment with assurances that the test configurations
conform to product definition and requirements.
– Comprehensive test procedures within test plans contain pass/fail criteria that can be leveraged
by any workgroup. Rigorous database rules help verify that each test procedure criteria is
executed only when certain conditions are met.
– Test configurations are tracked at the hardware revision level of each device, including
firmware (FW)/BIOS revisions, and at the software versions level for each component used
during software image download and test.
3
• Automated Test Environment—This enables test teams to download a products software image,
automatically install the software to replicate the manner in which a typical customer might install
it, automatically run test scripts, and then log results into the database. For a given product
configuration, hundreds of different software downloads, installations, and tests can be logged
with minimal time and effort.
• Quality Assurance—Software image development databases and tools enable a complete, end-to-
end, repeatable image test process. All HW, FW, and SW changes are systematically managed.
System checks are in place to help make sure that what gets built in any factory worldwide and
sent to the customer is the same as what was tested and qualified in engineering.
Types of Testing
HP executes a vast array of testing on Business Desktop PCs to help ensure that systems meet
important industry standards. Multiple engineering teams conduct various types of testing throughout
the product development cycle. This helps mitigate risk by providing needed checks and balances and
it helps to ensure a high HP standard of quality. These tests include, but are not limited to:
System Board Validation Testing
This testing focuses on the signal quality of high-speed interfaces, power delivery to system
components, and ACPI power management. The System board is put through stress testing to help
ensure that it will operate normally while running a variety of applications. Testing includes:
• Signal Integrity Testing—This helps ensure proper signal quality on all major interfaces on the
system board including front-side bus, memory bus, system clocks, USB (universal serial bus),
SATA, and PCI / PCI Express buses.
• Voltage Integrity Testing—This helps ensure that voltage regulators on the system board and in the
power supply remain within HP Business Desktop specifications while running stress level testing.
Additionally, voltage stress testing is conducted and measured on the various supported memory
and processor configurations.
• Power Management Testing—This helps ensure that the system will go into and come out of all
supported low-power states including S1, S3, S4, and S5. Additionally, power cycle testing is
conducted to simulate many years worth of power-up/power-down usage with a minimum of
10,000 consecutive no-fail cycles.
• Fan control—This helps ensure that the system fan(s) runs at required minimum thermal and
acoustic levels for customers’ needs.
Platform Validation Testing
This testing focuses on chipset validation, new architectural chipset features, the system board,
processor, system memory, system BIOS, and system-level hardware interoperability to help ensure
platform component stability prior to incorporation into an integrated desktop solution. Testing
includes:
• System BIOS Testing—This is comprehensive validation and qualification of the system BIOS.
Testing includes BIOS calls and functionality, supported BIOS features, and expected BIOS
interoperability with the platform hardware and software.
• System Board—This is comprehensive validation and qualification of the system board (PCB/PCA)
including testing of the chipset functionality/feature set, subsystem (i.e., CPU, Memory, Graphics,
Audio, PCI/PCIE, NIC, USB, IDE, LPC, etc.), and associated hardware components.
4
• Processor Qualification—This is comprehensive validation and qualification of each unique
processor, including all processor frequency changes and all processor stepping changes.
Processor qualifications are executed on all supported platforms.
• System Memory Qualification—This is comprehensive validation and qualification of each unique
memory module. System memory module qualifications are executed on each supported platform.
Mechanical Validation Testing
This testing focuses on environmental and Industrial Design factors for the chassis, power supply, and
system board. Mechanical testing includes form, fit and function tests; paint finish; durability tests; and
security tests to help ensure that the product can be assembled correctly. Additional testing includes:
• Shock and Vibration Testing—Units are subjected to random vibration and shock to help ensure
that the system functions correctly in harsh environments.
• Packaging Testing—Boxed systems are subjected to resonance, random vibration, and multiple
drops from various heights to help ensure that the packaging can withstand rough handling and
protect the system.
• Thermal Testing—Each system board, power supply, and integrated subsystem (memory, graphics,
audio, NIC, etc.) is tested in elevated ambient, maximum power, and minimum power situations to
help ensure that no component temperature specifications are exceeded.
• Environmental Testing—Testing is conducted at various temperatures, altitudes, and humidity
settings for extended periods of time to help ensure that the system design is reliable in many
environments.
• Acoustics Testing—This testing helps ensure quiet, reliable products. Acoustic evaluations are
performed with typical and power user applications: at idle, when the hard drive is being
accessed, and at tonal resonances (noises that could be perceived as adverse by the customer).
• Safety/Agency—Systems undergo extensive electromagnetic compatibility and safety testing to
help ensure that the product is safe to operate, is environmentally friendly, meets European
ergonomic fatigue standards and other various local regulations, does not interfere with radio or
TV reception, and is free from interference by external electrical effects.
• Power Supply Testing—This testing focuses on the power supply; it includes full electrical
verification, voltage regulation, output ripple and noise, stability, input and output transients, stress
margins for voltage and current, high-voltage isolation, electro-static discharge, and harmonic
current emissions.
• Reliability Testing—System and component level testing and modeling is geared towards
mitigating risk such as Mean Time Between Failures (MTBF), Strife, High and Low temperature
power cycling, long-term reliability, cold/warm boot, AC power on/off, and various other tests.
Module Functional Testing
This testing focuses on a wide variety of module subsystems to help ensure HP Business Desktop PC
standards are met prior to integration into a desktop solution. Module functional testing is conducted
on the graphics, audio, network controller, hard drive, optical drive, and input/output device
subsystems, as well as on numerous software applications. Testing includes, but is not limited to:
• Electric Signal Integrity Testing—This testing helps ensure that the proper signal quality is met on all
interfaces and layout routes through the specific module components.
• Mechanical Testing—Environmental and design ID testing helps to ensure the proper fit and
function and temperature variance tolerance for a module.
• Electro-Static Discharge Testing—This testing helps ensure that a module component meets
specifications for continued operation after being subjected to electro-static discharges at various
points on the device.
5
• Electro-Magnetic Conductance Testing—This testing helps ensure that a module is not susceptible to
and does not emit interfering radio waves.
• Power Management Testing—This testing helps ensure that a module goes properly into and out of
defined low-power states.
• Software Driver Testing—This testing helps ensure that the software drivers for a module perform as
required, deliver expected performance and features, and integrate into HP images to meet
customer needs and latest technology.
• Software Application Testing—As part of the total PC solution, HP offers select software
applications, including hardware-enabling applications (such as DVD playback) and productivity
type applications (such as Microsoft Office). Testing is conducted to help ensure that these HPspecific and industry standard applications offer the required features and integrate into HP
images.
• Video Timing Testing—This testing helps ensure proper graphics controller output for image display
on a variety of monitors by complying with video timing specifications for all supported resolutions
and refresh rates. Additionally, commercial HP monitors are tested in combination with the HP
graphics controller solutions to better ensure proper image display.
• Network Interface Testing—This testing helps ensure proper functionality of the network interface.
Testing includes, but is not limited to, the following: bit-error-rate, auto-negotiation, networking
protocol testing, and CRC (Cyclic Redundancy Check). Testing also verifies that packet loss
specifications are not exceeded and that network packet patterns are as designed.
System Integration Testing
This testing focuses on the integration and overall quality of the hardware and software module
components assembled into a complete desktop solution. Testing is conducted across a wide variety
of configurations.
• Image Integration Test helps verify:
– integration of pre-installed operating systems and software in all supported languages.
(~3 OSs, ~25 languages)
– operation of the system after using the restore solutions provided with our platforms
• System Compatibility Test helps verify:
– hardware and software compatibility of supported modules on new products. (combinations of
interdependent modules)
– functionality of integrated system hardware and its compatibility with alternate operating
systems (Linux, OS/2, Novell, etc. if supported)
– system hardware and pre-installed software compatibility with 3rd party hardware and
software products using primary and alternate operating systems. (~300 3rd party products)
– compatibility of the systems with commonly used accessibility products
– physical form and fit of standard and aftermarket modules on new products
– interoperability of HP Business Desktop PCs with other HP products
– compatibility by participating in variety of “plugfests” sponsored by HP and other component
vendors
6
• Localization Test
– represents interests of geographic areas around the world to the development team
– execute localization test plans to check for
o accurate translations
o appropriate language
o correct user interface
– functionally tests translated versions of products
• Module Integration Test
– provides acceptance testing to help ensure selection of high quality options
– verifies the functionality of supported peripherals and applications in a wide variety of
combinations of modules and platforms
Performance Testing
HP runs a number of popular, industry-standard and press-review benchmark tests to measure the
overall performance of new Business Desktop PCs. These tests target unique sub-systems as well as full
systems. Benchmarks are used to monitor the performance impact of design and configuration
changes. Representative system configurations are tested through several design phases up to the
production level. HP follows each benchmark’s guidelines for preparing/configuring the operating
system and other configuration settings prior to testing. Through these benchmark measurements, we
can determine how well the system’s performance in each design phase meets our product goals.
As benchmark updates are released, HP quickly transitions to using the latest versions. Archives for
previous test versions and the associated results are maintained for desktop products.
Benchmarks used include:
• Business Winstone
• Multimedia Content Creation Winstone
• SYSMark
• PCMark
• 3DMark
• WebMark
Desktop Certifications
Certification testing is a unique type of testing that supports the goal of obtaining certain shipping and
logo rights for Business Desktop PC solutions. Specific certification tests must be passed in order to
successfully gain these rights. Testing is conducted prior to submission to resolve any issues that would
impede passing. These types of certification testing include:
• Windows Hardware Quality Lab (WHQL) Testing and Certification—Passing this required
operating-system-level testing and certification allows HP to display the “Designed for Windows”
logo on a product. A Microsoft-controlled test suite is used as a validation tool prior to system
submission to Microsoft for certification.
7
• Section 508 Testing and Certification—Section 508 is set of rules developed, maintained, and
enforced by the Access Board and General Services Administration (GSA) that helps to ensure that
electronic and information technologies provide or support certain accessibility features or
products for people with disabilities. For Section 508, HP conducts audit-level testing to verify
conformance. HP publishes the Voluntary Product Accessibility Template (VPAT) on the HP
Accessibility web site for each product.
• Linux Testing and Certification—Select HP Business PCs are currently certified for Novell SUSE
LINUX and/or Red Hat Enterprise Linux. Additional or different Linux providers may be supported
on future products. Testing is conducted prior to submission using available industry tools and
utilities with the goal of helping to ensure a passing Linux configuration.
• Novell Testing and Certification—Select HP Business PCs are submitted for certification for use with
the Novell Netware Client.
Tested Configurations
With the multitude of components included in a PC, and the number of operating systems,
applications, and options offered, testing of every permutation becomes impossible. However, using
disciplined test-engineering methods, HP helps ensure good test coverage. Detailed understanding of
component interdependencies and testing technologies allows HP to create a set of test configurations
that are representative of many of our customers’ usage.
The hundreds of systems we use in the test process and the numerous well-controlled configurations
we build during development help maintain our high level of quality.
Hours of Testing
HP Business Desktop PCs are subjected to on average over 130,000 machine hours of testing. The
number of test hours varies as new system chipsets and system boards, chasses, video, audio and
other module components, and/or operating systems are incorporated into a desktop design. Table 1
shows the approximate test percentage allocation to qualify a business desktop PC with typical
architectural changes.
Table 1: Test coverage per area
Test area Estimated Percentage of
total test hours
System Board Validation 15%
Platform Validation 12%
Mechanical Validation 13%
Module Functional Testing 17%
System Integration Testing 40%
Performance & Certification Testing 3%
8
Observations during Testing
The result of this exhaustive testing is a detailed list of observations made by test engineers. The
observations are reported when the behavior of the PC is different than what is expected, based on
design specifications, product requirements, and an understanding of customer needs. These
observations are reviewed by the responsible HP staff and management team using robust database
tools and tracking processes, then HP determines the disposition for each observation.
After development is complete and the product is shipped to the end user, customer satisfaction is
closely monitored and customer-reported problems, requests, and comments are tracked. HP attempts
to resolve customer issues, and information is fed back to the development and test teams, thus
completing this closed-loop process. In this way, we help ensure that the Total Customer Experience
(TCE) is continuously improving.
On-going Qualifications
Testing does not stop after a new Business Desktop product completes initial testing and the new
Business Desktop computer is shipped. Rather, testing continues throughout the product lifecycle to
incorporate product updates including BIOS, module HW and SW, along with customer support
feedback. Additionally, test process improvements are constantly folded into the on-going
qualification process.
9
For More Information
In conclusion, the HP Business Desktop product development team values testing and believes that the
HP customer values testing. We feel strongly that HP’s level of testing leads to a higher quality
product, which benefits our customers. A Business Desktop PC is more than just an assembly of
various modules and components; it is the result of a concerted development and integration effort,
with checks and balances throughout the qualification cycle. It is a PC that the product development
team is proud to have carry the HP brand.
Additional information can be obtained through the HP website at www.hp.com
© 2004 Hewlett-Packard Development Company, L.P. The information
contained herein is subject to change without notice. The only warranties for
HP products and services are set forth in the express warranty statements
accompanying such products and services. Nothing herein should be construed
as constituting an additional warranty. HP shall not be liable for technical or
editorial errors or omissions contained herein.
372011-001, 06/2004
Loading...