1. Introduction

The large-scale system shall adopt distributed network architecture, and shall have openness and good scalability to adapt to the changing application environment and requirements; Each module in the system processes different types of data according to its division of labor, which should be relatively independent and interrelated at different levels, so as to realize mutual access and collaborative work; The system should also have good integration, need an effective component construction framework at the functional level, and have a unified data interaction platform at the component level.

Based on the above analysis, we choose CPCI bus as the data communication platform of distributed system. CPCI bus technology is the combination of PCI bus technology and mature European card assembly technology. In terms of electrical, logic and software functions, it is fully compatible with PCI standard, and breaks through the limitation of four slots of PCI standard. Compared with VME bus module, CPCI bus technology has the advantages of openness, easy expansion, high density and 99.999% high availability. Adopting CPCI bus technology and hardware interface design specification, it can use its multi module plug-in card design advantages to support multi service distributed processing, realize the seamless connection of modular data processing units, provide high-speed and reliable guarantee for distributed data exchange, and is very suitable as a distributed system service processing communication platform, It is also suitable for wide application in communication and embedded systems.

This paper presents a communication system design based on CPCI bus. The system adopts distributed network architecture to support the processing and data interaction of a variety of packet switching services. Firstly, the system structure and principle design are given. Aiming at the difficulties of cross bus communication of distributed business processing module, the message storage and address information maintenance strategy based on “drawer mechanism” is proposed. The implementation of key technologies such as data non-interference transmission is described. Finally, the technical summary and prospect are given.

2. Overall system design

2.1 system structure characteristics

The distributed system structure we designed is shown in Figure 1. Different equipment boards in the system independently process the corresponding service data, convert it into unified IP data for interworking, maintain their own routing tables, and independently complete data forwarding. The system distributes the interfaces with the specific service network to various equipment boards for standard access channel adaptation, and distributes various network data to each board for processing and forwarding, realizing the perfect combination of centralized configuration and distributed access and data processing.

In the CPCI distributed bus architecture, the backplane provides physical connection and circuit guarantee for bus switching, and the system slot on the backplane provides bus arbitration, clock distribution and restart of each board on the backplane; Simple interface board, intelligent slave device or bus control device can be placed on the peripheral slot. Each CPCI board has a processor and embedded real-time system. The processor adopts powerpc-860 of Motorola and 9054 and 9056 PCI bridge chips of PLX company to build an efficient and stable transmission bridge between powerpc-860 and CPCI bus. PLX 9054 / 9056 chip realizes the function of CPCI master control equipment, supports pci2.2 procedure, simplifies the design of connecting PowerPC, has good compatibility, and can be easily extended to 66MHz clock and 64bit PCI bus, especially PLX 9056 embedded bus arbiter, which can reduce the system scale and make the system more stable.

How to design a CPCI bus distributed communication system? What are the characteristics of the system?

Figure 1 structure diagram of data communication system

2.2 system resource sharing and information exchange

The system adopts the distributed architecture of single bus multiprocessor / multi operating system based on CPCI. Each board in the system has independent CPU and operating system, address and memory space, as well as independent I / O and interrupt, which can complete data operation independently. Each board can be regarded as a computer host. The topology of the distributed system is a fully connected network, and each node in the network can directly access other nodes; From the perspective of CPCI bus transmission, the boards in all slots are peer-to-peer and can act as a master to actively initiate bus transmission. For this bus based distributed architecture, we design a cross bus memory access mechanism to map the system memory or device memory (such as memory expansion card) of other boards in the system to the local address space, and then access the mapped memory in the same way as the system memory, In this way, each board can access the memory resources of other boards on the bus.

2.3 unified and standardized access interface

Heterogeneous networks are connected to the distributed system through standard channel adaptation. Non IP data such as voice, X.25 and serial port data are converted into IP data through the data adaptation module. The embedded real-time system of each board in the system processes and interacts with the data. Various heterogeneous networks are connected to the corresponding network devices in the embedded system. The network device driver calls the unified interface provided by CPCI bus driver to realize the data transmission between the real-time system and the bus. When sending data, the network device driver performs address conversion, data forwarding and interrupt generation through the bus drive control bridge chip to generate corresponding bus operations and send the data to the bus; When receiving data, the bus driver responds to the interrupt, receives the data of the corresponding address segment on the bus, and performs data analysis, address conversion, data forwarding, other interrupt generation and other operations in the interrupt service program. We adopt the Linux operating system, and its network system is mainly based on the socket mechanism of UNIX. The system protocol stack and driver transmit data through a special data structure (sk_buff). The data transmission process between real-time system kernel and CPCI bus is shown in Figure 2:

How to design a CPCI bus distributed communication system? What are the characteristics of the system?

Figure 2 data transmission flow chart

3. Key technologies

3.1 “drawer mechanism” for message storage

Each board in the system shares a CPCI bus. We propose a message storage strategy based on “drawer mechanism” to ensure non-interference transmission of data between boards. In the initialization stage when the board is added to the system, the system board assigns an independent PCI bus address interval to each board on the bus, and when other boards send data to it, they write the data to the specified address area. A board will receive data from different boards. In order to avoid the interference caused by each board sending data to the same base address, an independent read-write space of the same size is allocated to other boards in the same board address area. We name it “drawer”. In this way, the data from a board will be sent to its corresponding “drawer”, and each time the data will be stored in order rather than overwritten, so as to ensure the data processing time of the board. When the data length exceeds the remaining space of the drawer, it is stored from the beginning like a ring buffer. “Drawer mechanism” is shown in Figure 3. The box on the left represents different boards on the bus, and the address space of PCI bus is on the right. The corresponding address range of board B is from point a to point E, where the space between ab points is only used for data transmission from board a to B, and the space between BC points is only used for data transmission from board c to B, and so on.

How to design a CPCI bus distributed communication system? What are the characteristics of the system?

Figure 3 card data receiving “drawer”

Based on this message storage mechanism, we define several address tables to maintain address information related to data transmission. A static base address table is maintained on the system board, which records the pre allocated base address of the board on each card slot. Card address mapping table and transmission address offset table are maintained on all cards. The board address mapping table is a structure array. Each item in the array represents a card slot, which contains address information such as board name, card slot number, base address and address range for configuration during data transmission. Its data structure is as follows:

typedef struct _BUS_ADDR_MAPPING_INFO{

char board_name[BOARD_NAME_LENGTH];

int slot_number;

unsigned long base_addr;

unsigned long range;

}BUS_AddrMapping_Info, *P_BUS_AddrMapping_Info;

The transmission address offset table is an unsigned integer array used to record the address offset of each board during data transmission between boards. The initial value is zero. After each transmission, the address offset of the receiving board increases the current data transmission length. When the address space is insufficient to store the data to be transmitted, set the offset address to zero and write it again from the beginning of the area. The data structure is defined as follows:

u32 current_offset_table[NUM_OF_SLOT] = {0,0,0,0,0,0,0,0};

3.2 realization of data transmission

We define a data structure IPH (internal packet header), which contains attributes such as data type, length and source card slot number. Before transmitting data, the message is encapsulated as a packet header, so that the receiver can distinguish and process according to the data service type after parsing the packet header. The main IPH types include board configuration information, port registration information, routing information, unknown data type, etc. Define data structure IPH_ Attr distinguish different IPH_ Info type, which is located at the beginning of the packet. Its data structure is as follows:

typedef struct _IPH_ATTR {

u32 board_id; /*from which board*/

int iph_type; /*datagram type*/

unsigned long length; /*datagram length(without IPH)*/


Different data structures are defined for various types of IPH information, which are stored in the IPH in turn in the packet header_ After the attr structure.

When sending data, carry out IPH encapsulation on the data, select the destination PCI address according to the above board address mapping table, and then call the bus interface function to complete data transmission. The sender notifies the receiver of the transmission address and data length information by writing the mailbox register of the receiving board bridge chip, and generates an interrupt to trigger the reception. The PLX bridge chip supports the direct access of the local bus to the PCI bus. It has 8 mailbox registers. The first four can generate interrupts, each mailbox32 bits. The transmission address and data length information work together with mailbox I and mailbox I + 4 respectively. In this way, when the receiver receives two parameters, it will generate an interrupt for data reception [5]. This mechanism makes the receiving processing have four service windows and improves the system throughput.

When the mailbox of the PLX chip of the receiving board is written into the parameter, a local interrupt is generated to check the “drawer”. Before the interrupt is generated, the data has actually been sent to the target board. The interrupt service program maintains a data queue for the receiver. It reads the information in the mailbox, analyzes the address, finds the corresponding data, and sends it to the bottom half for processing. The IPH of the bottom half parses the packet to distinguish the data type. If it is configuration, port, routing and other information, it will be configured accordingly. If it is data information, it will be processed or forwarded.

To sum up, the system completes the mapping of PCI address space between boards through the “drawer mechanism” and the maintenance of several address tables. The board writes the data to the mapped address space, that is, it can transmit the data to the target board through the bus, realizing the cross bus memory access of the board; Customize the IPH packet header to distinguish data types, assist in data information management, complete the logic functions of routing maintenance and forwarding engine, and realize the non-interference transmission and effective communication management of data.

4. Summary and Prospect

The author’s innovation is: a distributed system design based on CPCI is given, and a message storage mechanism and address information maintenance strategy based on “drawer mechanism” are proposed. The CPCI based distributed system described in this paper can reach 64bit bus width and 264mb / s peak bandwidth. Each host in the system can independently complete data processing and communication, and can carry a variety of voice and data services. Users can also carry out data communication through the PSTN network connected to the voice service board and the Internet connected to the data service board It has great application prospects in military and other fields. In order to make the communication system more large-scale practical value, future work includes:

(1) Implement a set of easy to operate remote management system to complete the monitoring and deployment of communication services;

(2) Design CPCI interface cards supporting more business types, such as xDSL, H.264, etc;

(3) In the case of large external interference, strict performance test is carried out to prove that the system can meet the service requirements of carrier level.

Leave a Reply

Your email address will not be published. Required fields are marked *