RE: CERN IPMC question

Yuri_Ermoline · June 5, 2023, 11:32am

Hello Stefan,

You can find all the files from the project here:

http://ermoline.web.cern.ch/ermoline/IPMC/ipmc-project/

Please, have a look… In the ipmc-user/user_oem_examples.c

there is a HUB OEM command for the Power Entry module IQ65033 - the only I2C device on the MGT I2C bus.

When I run:

-bash-4.2$ ipmitool -I lan -H asml1c-stf0 -U “” -P “” -t 0x82 raw 0x2e 0x90 0x60 0x00 0x00

60 00 00 00

I’m reading ‘0’ that is not correct… something doesn’t work…

Can you look at the code when will have time, please?

Thank you,

Yuri

(Attachment IQ65033QGA12.pdf is missing)

Yuri_Ermoline · June 5, 2023, 12:17pm

Hi Stefan,

The project is also on the git now:

https://gitlab.cern.ch/atlas-l1calo/firmware/hubrod-ipmc

Thank you,

Yuri

haass · June 5, 2023, 12:53pm

Hello Yuri,

I had a quick look, are you sure you are suing the correct I2C address? Shouldn’t it rather be 0x58 instead of 0x2C? Please give that a try.

In any case, you should always check the return status of the I2C access command and report an error on the IPMC console if any of the fail. Also adding debug priontouts to your code would certainly help.

cheers,

Stefan

haass · June 5, 2023, 12:55pm

Hello yuri,

thanks, however I cannot access that repository, I can only see efex-ipmc. Maybe you didn’t add me to the members of the project?

cheers,

Stefan

Yuri_Ermoline · June 5, 2023, 2:05pm

Hello Stefan,

Thank you! I added you to the team on the Git – please, check.

I understand your “0x58 instead of 0x2C”.

For the LAPP IPMC I supplied 7 bit I2C address = 0x2C.

R/W bit was added by I2C_Write or I2C_Read command.

In your case - 0x58 – does it imply 2 addresses – for read and for write?

In “sensor_iq65033.xml” I only specify one i2c_addr.

You have to commands: i2c_dev_read and : i2c_dev_write – do they use the same address and i2c_dev_read just add “R” bit?

Please, clarify.

Thank you,

Yuri

haass · June 5, 2023, 3:53pm

Hello Yuri,

haass · June 5, 2023, 7:23pm

Hello Yuri,

thanks, the access to the repository works now.

cheers,

Stefan

Yuri_Ermoline · June 5, 2023, 7:43pm

Hi Stefan,

I correct the i2c address in “sensor_iq65033.xml”

0x0158

and in oem command

unsigned short i2c_addr = 0x0158; // IQ60533 address on MGT_I2C

(Git updated)

However, still reading “00” when run command my oem command:

-bash-4.2$ ipmitool -I lan -H asml1c-stf0 -U “” -P “” -t 0x82 raw 0x2e 0x90 0x60 0x00 0x00

60 00 00 00

Thank you,

Yuri

haass · June 6, 2023, 9:23am

Hello Yuri,

can you please try to read the temperature register instead, since the current could actually really be zero,

Failing that, can you please check the return status of the I2C access functions in your code and print an error to the console if it fails. You can have a look at the existing sensors drivers for examples. In addition, as I already suggested before, you should add debug printouts to your code.

cheers,

Stefan

Yuri_Ermoline · June 6, 2023, 9:53am

Hi Stefan,

Tried to read temperature register – with the same result - “00”.

(and current is not zero as far as I know with the LAPP IPMC).

To print errors I need to use SOL (there is no any serial output from the IPMC on the HUB).

So, need to understand how to do this.

When I got the IPMC from Markus, I didn’t register it in the LAN DB.

Shall I register it to use SOL? What is the MAC address?

Markus gave me : 80-03-36-00-40-00

When I run OEM your command “oem_get_macaddr“ to read the MAC, I’m getting:

-bash-4.2$ ipmitool -I lan -H asml1c-stf0 -U “” -P “” -t 0x82 raw 0x2e 0x4F 0x60 0x00 0x00

60 00 00 d0 40 00 36 d3 80

So, what is correct?

Thank you,

Yuri

haass · June 6, 2023, 2:37pm

Hello Yuri,

concerning the MAC address, the value returned by the OEM command should be the correct in the EEPROM, i.e. 80-D3-36-00-40-D0. Maybe it was overwritten?

Do you have a direct connection to the IPMC Ethernet port on the front-panel? In this case it would be easier to use a USB-Ethernet dongle on a PC to connect to it. If you want to connect it to the GPN, you will either have to ask for a fixed IP (and set it in the XML) or enable DHCP in the IPMC.

In any case, I believe it’s essential to have access to the IPMC console, otherwise it becomes very difficult to debug anything.

cheers,

Stefan

Yuri_Ermoline · June 6, 2023, 7:01pm

Hi Stefan,

I registered the IPMC (it is in the L1Calo STF lab):

ACML1C-STF0-HUB00-IPMC

IP: 10.11.30.225

MAC: 80-D3-36-00-40-D0

DHCP Enabled

It replies to ping.

I enabled SOL by running two commands, suggested by Paul and then I got:

-bash-4.2$ ipmitool -I lanplus -H ACML1C-STF0-HUB00-IPMC.CERN.CH -C 0 -U “” -P “” sol payload enable 5 1

-bash-4.2$ ipmitool -I lanplus -H ACML1C-STF0-HUB00-IPMC.CERN.CH -C 0 -U “” -P “” sol activate

[SOL Session operational. Use ~? for help]

<_>: BMR-A2F Firmware (v1.3.0)

<_>: Reset type: hard, reset cause: software

MAC Address loaded from EEPROM [80:D3:36:00:40:D0]

<_>: Perform Power-on-Self Testing.

<_>: Device type: A2F200M3F-CS288

<_>: MSS clock frequency: 80 MHz

<_>: Fabric clock frequency: 20 MHz

<_>: Fast delay calibration: 2665 cycles per 100uS

<_>: FPGA design: BMR-A2F-ATCA, version 4.0.0

<_>: Flash: 256 KB (00000000 - 00040000)

<_>: SRAM: 6I2C dev read error, I2C address: 0158

I2C dev read error, I2C address: 0158

… endless

So:

Any comments?
How I can stop this endless messages?

Thank you,

Yuri

haass · June 7, 2023, 7:11am

Hello Yuri,

the messages indicate that there is no response from at the specified address. They certainly come from the sensor driver, I suggest that you temporarily comment the IQ65033 sensors out in the XML and use the OEM command for the tests.

Can you please add debug messages to your OEM code, in particular printing the return status of the functions. You could also make the code more flexible by adding the possibility to pass in the address as a parameter for the OEM command.

Can you also please point me to the schematics of the Hub modules?

cheers,

Stefan

Yuri_Ermoline · June 7, 2023, 9:50am

Hi Stefan,

OK, disabling the sensor driver stops the endless messages.

Will look into debug messages.

HUB schematics is here:

https://web.pa.msu.edu/hep/atlas/l1calo/hub/hardware/drawings/Circuit_Diagrams/

also attached.

Thank you,

Yuri

(Attachment circuit_diagrams.pdf is missing)

Yuri_Ermoline · June 7, 2023, 7:54pm

Hello Stefan,

I aged a few debugging messages:

static cmd_ret_t oem_iq65033_read_curr(msg_header_t *header, unsigned char *rep, unsigned char *compcode)

{

debug_printf(“Reading the current register:\n”);

*compcode = IPMI_SUCCESS;

unsigned short i2c_addr = 0x0158; // IQ60533 address on MGT_I2C

unsigned char dataPointer = 0x21; // Current Register Data_Pointer

unsigned char i2c_rep; // Read value

// Write Data_Pointer

// char i2c_dev_write(unsigned short addr, unsigned char *pdata, unsigned char size)

if (i2c_dev_write(i2c_addr, &dataPointer, 1))

{debug_printf(“i2c_dev_write problem\n”);}

// Read Data

// char i2c_dev_read(unsigned short addr, unsigned char *pdata, unsigned char size)

if (i2c_dev_read(i2c_addr, &i2c_rep, 1))

{debug_printf(“i2c_dev_read problem\n”);}

// Add the i2c_rep byte to the IPMI reply - reply data buffer 5 bytes

rep[4] = i2c_rep;

return 0;

}

Now I got after running

-bash-4.2$ ipmitool -I lan -H asml1c-stf0 -U “” -P “” -t 0x82 raw 0x2e 0x90 0x60 0x00 0x00

60 00 00 00:

On the message console:

Reading the current register:

I2C dev read error, I2C address: 0158

i2c_dev_read problem

It doesn’t tell me more than I got before.

Can you just tell me what debugging to add to get useful information for you?

Thank you,

Yuri

haass · June 8, 2023, 6:40am

Hello Yuri,

at least we know now that the device responds to write cycles (this one does not give an error), so it seems the problem is with the read cycle. Could you please try the following code for the read:

if (i2c_io(i2c_addr | I2C_READ | I2C_START | I2C_STOP, &i2c_rep, 1) != I2C_OK) {
// print error message
}

You may have to include the header file i2c.h

Let me know if this helps.

cheers,

Stefan

Yuri_Ermoline · June 8, 2023, 9:42am

Hi Stefan,

Did, now:

-bash-4.2$ ipmitool -I lan -H asml1c-stf0 -U “” -P “” -t 0x82 raw 0x2e 0x90 0x60 0x00 0x00

60 00 00 00

On the message console:

Reading the current register:

(No other messages)

Cheers,

Yuri

haass · June 8, 2023, 10:00am

Hello Yuri,

can you please change to offset to read the temperature instead of the current? Also please initialize the i2c_rep variable, e.g. with 0xAA. It would also be good to add a printout of i2c_rep after the read in the OEM function.

cheers,

Stefan

Yuri_Ermoline · June 8, 2023, 12:28pm

Hi Stefan,

static cmd_ret_t oem_iq65033_read_curr(msg_header_t *header, unsigned char *rep, unsigned char *compcode)

{

debug_printf(“Reading the current register:\n”);

*compcode = IPMI_SUCCESS;

rep[0] = 0xAA;

unsigned short i2c_addr = 0x0158; // IQ60533 address on MGT_I2C

// unsigned char dataPointer = 0x21; // Current Register Data_Pointer

unsigned char dataPointer = 0x28; // Temperature Register Data_Pointer

unsigned char i2c_rep; // Read value

// Write Data_Pointer

// char i2c_dev_write(unsigned short addr, unsigned char *pdata, unsigned char size)

if (i2c_dev_write(i2c_addr, &dataPointer, 1))

{debug_printf(“i2c_dev_write problem\n”);}

// Read Data

// char i2c_dev_read(unsigned short addr, unsigned char *pdata, unsigned char size)

// if (i2c_dev_read(i2c_addr, &i2c_rep, 1))

// {debug_printf(“i2c_dev_read problem\n”);}

if (i2c_io(i2c_addr | I2C_READ | I2C_START | I2C_STOP, &i2c_rep, 1) != I2C_OK) {

// print error message

}

debug_printf(“i2c_rep: %02xh \n”, i2c_rep);

// Add the i2c_rep byte to the IPMI reply - reply data buffer 5 bytes

rep[4] = i2c_rep;

return 0;

}

-bash-4.2$ ipmitool -I lan -H asml1c-stf0 -U “” -P “” -t 0x82 raw 0x2e 0x90 0x60 0x00 0x00

60 00 00 00

Reading the current register:

i2c_rep: 00h

Cheers,

Yuri

haass · June 8, 2023, 4:41pm

Hello Yuri,

I just realized that the IQ60533 seems to use exactly the same I2C protocol and registers as the PIM400KZZ that we use on our boards and which works perfectly with the CERN IPMC. Can you therefore please replace the two I2C function calls (write then read) in your OEM code with a single function call to i2c_dev_read_reg? The code from the PIM400KZ driver is shown below:

static char inline sensor_pim400_read_sensor(unsigned char reg, unsigned char *val)
{
*val = 0xFF; // unrealistic value to mark an error
if (i2c_dev_read_reg(PIM400KZ_I2C_ADDR, reg, val, 1)) return I2C_ERROR;
#ifdef DEBUG
debug_printf(“PIM400KZ @ 0x%03x, register 0x%02x: data = %d\n”, PIM400_I2C_ADDR, reg, *val);
#endif
return 0;
}

You will obviously have to replace the PIM400KZ_I2C_ADDR define with the address of your device (0x0158). In case this works, you will be able to use the PIM400KZ sensor driver that is in the IPMC distribution with very minor modifications.

cheers,

Stefan