Slow PCI performance on Intel Xeon server
I'm in the process of porting some of my company's software to Linux. I have an application that loads an FPGA image at runtime over the PCI bus on to our custom hardware.
The application works fine on my development PC, which is an Intel Core 2 Duo PC. The FPGA load takes about 25 seconds, about the same as the Windows port on the same hardware. However, on an Intel Xeon server, the same load operation takes 5 minutes.
Both systems are running RHEL 5.3 x86_64 (kernel 2.6.18-el5). All of the software is the same, the only difference is the hardware. I've attached a PCI analyzer to see if I could find anything unusual. Everything looks fine to me, except for the time between successive PCI writes. On the Xeon server, successive writes are spaced much further apart than on the Core 2 PC - the difference is an order of magnitute (I've forgotten the exact numbers).
The drivers I'm using are provided by a third party: Jungo. I'm using v10.11 of their x86_64 Linux driver.
I suspect that this is a software issue, possibly related to the driver that interacts with the PCI bus interface. The Xeon processor is a Nehalem part, W5590. Red Hat claims that it is supported, but I'm not sure how well supported it is. Their knowledge base article is: http://kbase.redhat.com/faq/docs/DOC-15262.
I'm looking for suggestions that will help me identify the problem. I'm new to Linux, but I'm not afraid to dig into the kernel code if I have to.
Thanks
Comments
-
What Xeon processor are you running? How many cores? What is the CPU speed (GHz)? What is the motherboard you are using? How is the BIOS configured? All of these things will contribute to system performance, and/or performance problems, along with other stuff such as other hardware on the system (IRQ and DMA conflicts come to mind), etc.0
Categories
- All Categories
- 178 LFX Mentorship
- 178 LFX Mentorship: Linux Kernel
- 772 Linux Foundation IT Professional Programs
- 382 Cloud Engineer IT Professional Program
- 175 Advanced Cloud Engineer IT Professional Program
- 75 DevOps IT Professional Program - Discontinued
- 7 DevOps & GitOps IT Professional Program
- 102 Cloud Native Developer IT Professional Program
- 7.6K Training Courses & Learning Paths
- 9 AI & ML Training
- 1 Blockchain & Decentralized Identity Training
- 28 Cloud & Containers Training
- 3 Cybersecurity Training
- 2 DevOps & Site-Reliability Training
- 1 Linux Kernel Development Training
- 2 Networking Training
- 2 Open Source Best Practice Training
- 5 System Administration Training
- 1 System Engineering Training
- 4 Web & Application Development Training
- 797 Hardware
- 202 Drivers
- 68 I/O Devices
- 37 Monitors
- 96 Multimedia
- 173 Networking
- 91 Printers & Scanners
- 91 Storage
- 771 Linux Distributions
- 81 Debian
- 68 Fedora
- 23 Linux Mint
- 13 Mageia
- 24 openSUSE
- 151 Red Hat Enterprise
- 31 Slackware
- 13 SUSE Enterprise
- 356 Ubuntu
- 469 Linux System Administration
- 31 Cloud Computing
- 73 Command Line/Scripting
- Github systems admin projects
- 101 Linux Security
- 79 Network Management
- 101 System Management
- 46 Web Management
- 146 Mobile Computing
- 20 Android
- 111 Development
- 1.2K New to Linux
- 1K Getting Started with Linux
- 403 Off Topic
- 126 Introductions
- 33 Study Material
- 1K Programming and Development
- 310 Kernel Development
- 715 Software Development
- 1K Software
- 416 Applications
- 182 Command Line
- 5 Compiling/Installing
- 71 Games
- 320 Installation
- Archived
- 183 Small Talk
- 2 LFD140 Class Forum
- 1.4K LFS258 Class Forum
Upcoming Training
-
August 20, 2018
Kubernetes Administration (LFS458)
-
August 20, 2018
Linux System Administration (LFS301)
-
August 27, 2018
Open Source Virtualization (LFS462)
-
August 27, 2018
Linux Kernel Debugging and Security (LFD440)