Slow PCI performance on Intel Xeon server

ijwed

I'm in the process of porting some of my company's software to Linux. I have an application that loads an FPGA image at runtime over the PCI bus on to our custom hardware.

The application works fine on my development PC, which is an Intel Core 2 Duo PC. The FPGA load takes about 25 seconds, about the same as the Windows port on the same hardware. However, on an Intel Xeon server, the same load operation takes 5 minutes.

Both systems are running RHEL 5.3 x86_64 (kernel 2.6.18-el5). All of the software is the same, the only difference is the hardware. I've attached a PCI analyzer to see if I could find anything unusual. Everything looks fine to me, except for the time between successive PCI writes. On the Xeon server, successive writes are spaced much further apart than on the Core 2 PC - the difference is an order of magnitute (I've forgotten the exact numbers).

The drivers I'm using are provided by a third party: Jungo. I'm using v10.11 of their x86_64 Linux driver.

I suspect that this is a software issue, possibly related to the driver that interacts with the PCI bus interface. The Xeon processor is a Nehalem part, W5590. Red Hat claims that it is supported, but I'm not sure how well supported it is. Their knowledge base article is: http://kbase.redhat.com/faq/docs/DOC-15262.

I'm looking for suggestions that will help me identify the problem. I'm new to Linux, but I'm not afraid to dig into the kernel code if I have to.

Thanks

woboyle

What Xeon processor are you running? How many cores? What is the CPU speed (GHz)? What is the motherboard you are using? How is the BIOS configured? All of these things will contribute to system performance, and/or performance problems, along with other stuff such as other hardware on the system (IRQ and DMA conflicts come to mind), etc.