We are currently using a LAPP IPMC with our LArC board with 4 AMC slots. The IPMC card from LAPP is no longer being made, so we would like to check whether the CERN-IPMC can be made to work in the same environment as the LAPP IPMC.
The CERN-IPMC placed on the LArC board is able to turn on the Carrier 12V, along with the RTM PWR (controlled by my user code). It is able to read sensors and send them to the shelf as well on the LArC board.
However, our CERN-IPMC is unable to turn on 12V PWR for our OTC AMC board plugged into our LArC Board. This AMC board works fine with our LAPP IPMC.
To setup the CERN-IPMC, the configuration xml file was edited as stated in “https://gitlab.cern.ch/ep-ese-be-xtca/ipmc-project#amc-slots”
In this configuration, we needed to invert “PS” and “EN” to get the management power to turn on. The configuration file used in linked here.
When we plug the AMC board into the carrier, the FRU associated with the AMC slot appeared on the shelf after approx. 15s along with the blue+red light indicating MGT power is on, though the FRU the shelf saw disappeared almost immediately. The CERN-IPMC was not able to give 12V PWR to the card when the handle was closed (which isn’t a surprise since the FRU went away).
Is the CERN-IPMC expected to be able to handle and see AMC slots? If so, is there anything we can do on our end to debug why this isn’t working?
Dear Mars,
Julian is sadly still on sick leave. And I am not very familiar with the use of AMCs.
The CERN IPMC is fully supporting AMCs, as well as an i-RTM. Those functionalities are tested on the IPMC Tester before we distribute the cards.
Did you configure and connect the Serial Debug Interface in order to get the debug/error messages from the IPMC when you are connecting your AMC? If you haven’t done so yet, I would strongly recommend doing so.
Have you looked through other topics on AMCs in this Discourse forum? I don’t know if other people/projects have more experience on the use of AMC and would be able to help.
Once you have narrowed down the problem further, we could raise a support issue with PigeonPoint.
Unfortunately, that’s all I can tell you right now.
Cheers,
Ralf.
PS: I could not read your config file - HTTP 404.
Dear Ralf,
Thank you for your response.
Unfortunately, we are unable to connect to the SDI. The LAPP IPMC has SDI ports directly on the IPMC card. These ports are not present on the CERN IPMC. We also cannot solder wires connecting to the SDI on the LArC board as the FPGA is in the way.
I have not found other relevant topics on AMCs in this Discourse forum. We will continue to do what we can to debug locally, but any advice will be helpful.
Kind Regards,
Mars
Dear Mars,
Sorry for my late reply, I am recovering from a long sick leave. Concerning your issue with AMCs, could you please give me more details: which firmware are you using, which version … ?
Concerning the SDI port, you can redirect it to the serial over lan interface that would give you a way to monitor it through the ethernet port.
Best regards,
Julian
Dear Julian,
Thank you for responding, I hope you’re well.
For SOL, we are having trouble connecting. I type in
ipmitool -I lanplus -H 192.168.1.22 sol activate -vv
And I receive the error
> RAKP 4 message has invalid integrity check value
If I instead try to go through the shelf
ipmitool -I lanplus -H 192.168.1.224 -U “” -P “” -t 0x88 sol activate -vv
The error becomes
RAKP 2 message indicates an error : unauthorized name
As for the firmware, the AMC’s do not turn on correctly using recently compiled code via the compile.py script. If this is not the firmware you’re asking about, how would I go about finding the firmware version?
Kind Regards,
Mars
Hi Mars,
Not sure that this corresponds to what you are observing, but we are using the following user, password, and option to activate the SOL:
ipmitool -I lanplus -H 192.168.3.10 -U “soluser” -P “solpasswd” -k gkey sol activate
Would you mind trying and letting me know if this helps in your case?
As for the AMC problem, I guess Julian would need to know what IPMC frmware AND what AMC firmware you are using …
Do you still have no power on your AMC?
Cheers,
Ralf.
Hi Ralf,
This worked, but I needed to use payload enable beforehand! I was able to start SOL using
ipmitool -I lanplus -H 192.168.1.22 -k gkey sol payload enable
ipmitool -I lanplus -H 192.168.1.22 -k gkey sol activate
From the IPMC output, I obtain
<_>: BMR-A2F Firmware (v1.3.0), AMC carrier edition.
<_>: Pentair Technical Products, Inc. (c) Copyright 2004-2016.
<_>: Reset type: hard, reset cause: power failure
<_>: Perform Power-on-Self Testing.
<_>: Device type: A2F200M3F-CS288
<_>: MSS clock frequency: 80 MHz
<_>: Fabric clock frequency: 20 MHz
<_>: Fast delay calibration: 2665 cycles per 100uS
<_>: FPGA design: BMR-A2F-AMCc, version 4.0.0
<_>: Flash: 256 KB (00000000 - 00040000)
<_>: SRAM: 64 KB (20000000 - 20010000)
The AMC is an OTC, developed before LATOMEs existed. The CERN MMC Mezzanine chip used on the IPMB bus is updated to around 2016 code.
While testing the AMC, we found best configuration so far is when we invert MPEN/MPGOOD PWREN/PWRGOOD signals in config.xml. This allows the MGT power to turn on, however we still cannot get the 12V to turn on.
In the console, we are obtaining the errors
<L>: <E> iface_ipmb_send_message_ipmb: i2c NACK
<E> iface_ipmb_send_message_ipmb: i2c NACK
<E> iface_ipmb_send_message_ipmb: i2c NACK
<E> iface_ipmb_send_message_ipmb: i2c NACK
<E> iface_ipmb_send_message_ipmb: i2c NACK
<L>: IPMB-L error: IPMC 74: no acknowledge
We have tried inverting the IPMBEN line in config.xml, but there was no change to the output.
Hi Mars,
I am glad to hear that the “ipmitool sol activate” finally worked for you.
I am not sure what the OTC is, but it has a CERN MMC, which is what we emulating when we are tetsing the IPMC cards. Maybe Julian has an idea on what could be wrong with switching on the AMC?
Cheers,
Ralf.
Hi @mlyukova,
There is an issue on old version of the CERN-MMC managing i2c conflicts. This is something a solved on our tester and I can try to extract a patch that, I hope, will solve you issue.
Cheers,
Julian
Hi Ralf, Julian
Thank you for your help so far, the NACK turned out to be an error on our end. We had <AMC site="1"> with PhysicalPort 2 in the config.xml file. It turns out the IPMB address of our AMC in Port 2 was 0x7C rather than 0x74.
We are now using <AMC site="6"> and PhysicalPort 2 . The CERN IPMC is able to correctly boot the AMC MGT power and 12V power with the handle closed with no issue.
For reference, our working config is
<AMC site="6">
<PhysicalPort>2</PhysicalPort>
<MaxCurrent>6.0</MaxCurrent>
<PowerGoodTimeout>300</PowerGoodTimeout>
<DCDCEfficiency>85</DCDCEfficiency>
<Invert>
<Pin>MPEN</Pin>
<Pin>MPGOOD</Pin>
<Pin>PWREN</Pin>
<Pin>PWRGOOD</Pin>
<Invert>
</AMC>
Where our power configuration for the LArC board is
<MaxCurrent>32</MaxCurrent>
<MaxInternalCurrent>7</MaxInternalCurrent>
Everything works as expected, all 4 AMC board are able to be plugged in and communicated with. They also show up in the shelf as well.
Thanks for the help. Kind Regards,
Mars
1 Like
Hi Mars,
Thanks. That is great news!
I am glad it now works for you.
Cheers,
Ralf.