IPMC Compilation not working

Software
1

Dear IPMCers,

I am trying to compile the ipmc-project gitlab repo files, and the script does not seem to complete :

(py2) mbenoit@pc-office-mb:~/projects/IPMC/ipmc-project$ python compile.py 
CERN Username: mbenoit
CERN Password: 
--> Clone ipmc-dev git repository
--> Git clone successfully finished
--> Configure IPMC source code
.Traceback (most recent call last):
  File "compile.py", line 296, in <module>
    main(USERNAME, PASSWORD)
  File "compile.py", line 264, in main
    link = CERNSSORequestInstance.post_stream(URL, data = values, files = files)
  File "compile.py", line 120, in post_stream
    for chunk in r.iter_content(chunk_size=1):
  File "/home/mbenoit/anaconda3/envs/py2/lib/python2.7/site-packages/requests/models.py", line 756, in generate
    raise ChunkedEncodingError(e)
requests.exceptions.ChunkedEncodingError: ('Connection broken: IncompleteRead(0 bytes read)', IncompleteRead(0 bytes read))

same in python3 :

(py3) mbenoit@pc-office-mb:~/projects/IPMC/ipmc-project$ python compile.py 
CERN Username: mbenoit
CERN Password: 
--> Clone ipmc-dev git repository
--> Git clone successfully finished
--> Configure IPMC source code
.Traceback (most recent call last):
  File "/home/mbenoit/anaconda3/envs/py3/lib/python3.9/site-packages/urllib3/response.py", line 697, in _update_chunk_length
    self.chunk_left = int(line, 16)
ValueError: invalid literal for int() with base 16: b''

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/mbenoit/anaconda3/envs/py3/lib/python3.9/site-packages/urllib3/response.py", line 438, in _error_catcher
    yield
  File "/home/mbenoit/anaconda3/envs/py3/lib/python3.9/site-packages/urllib3/response.py", line 764, in read_chunked
    self._update_chunk_length()
  File "/home/mbenoit/anaconda3/envs/py3/lib/python3.9/site-packages/urllib3/response.py", line 701, in _update_chunk_length
    raise InvalidChunkLength(self, line)
urllib3.exceptions.InvalidChunkLength: InvalidChunkLength(got length b'', 0 bytes read)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/mbenoit/anaconda3/envs/py3/lib/python3.9/site-packages/requests/models.py", line 760, in generate
    for chunk in self.raw.stream(chunk_size, decode_content=True):
  File "/home/mbenoit/anaconda3/envs/py3/lib/python3.9/site-packages/urllib3/response.py", line 572, in stream
    for line in self.read_chunked(amt, decode_content=decode_content):
  File "/home/mbenoit/anaconda3/envs/py3/lib/python3.9/site-packages/urllib3/response.py", line 793, in read_chunked
    self._original_response.close()
  File "/home/mbenoit/anaconda3/envs/py3/lib/python3.9/contextlib.py", line 137, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/home/mbenoit/anaconda3/envs/py3/lib/python3.9/site-packages/urllib3/response.py", line 455, in _error_catcher
    raise ProtocolError("Connection broken: %r" % e, e)
urllib3.exceptions.ProtocolError: ("Connection broken: InvalidChunkLength(got length b'', 0 bytes read)", InvalidChunkLength(got length b'', 0 bytes read))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/mbenoit/projects/IPMC/ipmc-project/compile.py", line 296, in <module>
    main(USERNAME, PASSWORD)
  File "/home/mbenoit/projects/IPMC/ipmc-project/compile.py", line 264, in main
    link = CERNSSORequestInstance.post_stream(URL, data = values, files = files)
  File "/home/mbenoit/projects/IPMC/ipmc-project/compile.py", line 120, in post_stream
    for chunk in r.iter_content(chunk_size=1):
  File "/home/mbenoit/anaconda3/envs/py3/lib/python3.9/site-packages/requests/models.py", line 763, in generate
    raise ChunkedEncodingError(e)
requests.exceptions.ChunkedEncodingError: ("Connection broken: InvalidChunkLength(got length b'', 0 bytes read)", InvalidChunkLength(got length b'', 0 bytes read))

My undestanding of the script is that is sends some zip file away from compilation, but does not get the proper file back.

Any idea what is going on ?

Thanks in advance,

Mathieu Benoit

2

Hi Mathieu,

Sorry for the inconvenience. The web service platform, OKD4, moved from python 3.8 to 3.9, which required a very small adaptation to our code. I just made that adaptation, and it works again for me. Could you please try it again?

Cheers,
Ralf.

3

Hi Ralf, thanks for taking a look at this.

It seems to work better and goes through, but I still have this error at the end :

ipmc-core/app/master/mainfru.c:741: warning: unused variable 'pmon_pgooda_fault_signal'
/usr/local/arm-toolchain/bin/arm-none-eabi-ar: creating ipmc-core/app/libcompat/libcompat.a
/usr/local/arm-toolchain/bin/arm-none-eabi-ar: creating ipmc-core/net/libnet.a
/usr/local/arm-toolchain/bin/arm-none-eabi-ar: creating ipmc-core/hal/libhal.a
/usr/local/arm-toolchain/bin/arm-none-eabi-ar: creating ipmc-core/lib/lib.a
/usr/local/arm-toolchain/bin/arm-none-eabi-ar: creating ipmc-core/hal/actel/libactel.a
crc32 for the cold data is 0xfcb0e6ed (7 vars total)
compat_record checksums: 0x56 0x81
File name: ipmc-core/hal/firmware.bin  size: 134288
Checksum section offset: 000004
Verify post_sram.c code
File name: ipmc-core/boot/boot.bin  size: 18072
Checksum section offset: 0002A0
Unused firmware flash: 86896
Unused boot loader flash: 6504
Making nvm blocks  
Loading .........Traceback (most recent call last):
  File "/home/mbenoit/projects/IPMC/cern-ipmc-development/gfex-ipmc-project1-4-1/compile.py", line 288, in <module>
    main(USERNAME, PASSWORD)
  File "/home/mbenoit/projects/IPMC/cern-ipmc-development/gfex-ipmc-project1-4-1/compile.py", line 256, in main
    link = CERNSSORequestInstance.post_stream(URL, data = values, files = files)    
  File "/home/mbenoit/projects/IPMC/cern-ipmc-development/gfex-ipmc-project1-4-1/compile.py", line 132, in post_stream
    strVar = strVar + chunk.decode("utf-8")
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe2 in position 0: unexpected end of data
4

Nervermind, I update the compile.py in our flavor to the latest one, I can build the image.

Thanks !

(py3) mbenoit@pc-office-mb:~/projects/IPMC/cern-ipmc-development/gfex-ipmc-project1-4-1$ python compile.py 
CERN Username: mbenoit
CERN Password: 
--> Clone ipmc-dev git repository
--> Git clone successfully finished
--> Configure IPMC source code
FRU information:   
        DONE: FRU file initialized
        DONE: Board information [FRU]
        DONE: Product information [FRU]
        DONE: LEDs description [FRU]
        DONE: Carrier information [FRU]
        DONE: General carrier power [FRU]
        DONE: fru-info binary file generated
                   
LAN configuration: 
        DONE: MAC address [0A:0A:0A:0A:0A:86]
        DONE: IP address [192.168.1.34]
        DONE: Netmask address [255.255.255.0]
        DONE: Gateway address [192.138.1.3]
        DONE: DHCP Enabled
        DONE: Default IP source (DHCP)
        DONE: MAC based DHCP enabled
                   
ATCA configuration:
        DONE: Device ID [0x12]
        DONE: Device Revision [0x00]
        DONE: Manufacturer ID [0x000060]
        DONE: Product ID [0x1236]
        DONE: Handle switch [Active: SIGNAL_LOW]
        DONE: SDI interface has been connected to port 0 [baudrate: 115200]
        DONE: PI interface has been connected to port 1 [baudrate: 115200]
                   
Power configuration:
        DONE: Power sequences configured
                   
Sensor configuration:
        DONE: SDR file initialized
        DONE: Device locator ID "IPMCv4" [SDR #0]
        DONE: ATCA Hotswap sensor [SDR #1]
        DONE: IPMB Sensor [SDR #2]
        DONE: Version change Sensor [SDR #3]
Not supported anymore
        DONE: Sensor "ZYNQ temp." added [SDR #4]
        DONE: Sensor "AD7417 temp." added [SDR #5]
        DONE: Sensor "PowerModule temp." added [SDR #6]
        DONE: Sensor "MiniPods temp." added [SDR #7]
                   
SDR binary:        
final formula: (100*x + 0E0)E-2
final formula: (100*x + 0E0)E-2
final formula: (100*x + 0E0)E-2
final formula: (100*x + 0E0)E-2
--> Configuration process successfully finished
--> Compile IPMC source code
make: Entering directory '/tmp/comp-Und0y2uCln'
ipmc-core/app/master/app.c: In function 'app_main':
ipmc-core/app/master/app.c:568: warning: unused variable 'dcdc_signal'
ipmc-sensors/sensor_spi.c: In function 'initialize_sensor_spi':
ipmc-sensors/sensor_spi.c:178: warning: 'wr_value' is used uninitialized in this function
ipmc-core/app/master/mainfru.c: In function 'mainfru_main_loop_callback':
ipmc-core/app/master/mainfru.c:747: warning: unused variable 'pmon_ab_fault_asserted'
ipmc-core/app/master/mainfru.c:745: warning: unused variable 'pmon_alarmb_fault_signal'
ipmc-core/app/master/mainfru.c:744: warning: unused variable 'pmon_alarma_fault_signal'
ipmc-core/app/master/mainfru.c:742: warning: unused variable 'pmon_pgoodb_fault_signal'
ipmc-core/app/master/mainfru.c:741: warning: unused variable 'pmon_pgooda_fault_signal'
/usr/local/arm-toolchain/bin/arm-none-eabi-ar: creating ipmc-core/app/libcompat/libcompat.a
/usr/local/arm-toolchain/bin/arm-none-eabi-ar: creating ipmc-core/net/libnet.a
/usr/local/arm-toolchain/bin/arm-none-eabi-ar: creating ipmc-core/hal/libhal.a
/usr/local/arm-toolchain/bin/arm-none-eabi-ar: creating ipmc-core/lib/lib.a
/usr/local/arm-toolchain/bin/arm-none-eabi-ar: creating ipmc-core/hal/actel/libactel.a
crc32 for the cold data is 0x52f09efe (7 vars total)
compat_record checksums: 0x56 0x81
File name: ipmc-core/hal/firmware.bin  size: 134288
Checksum section offset: 000004
Verify post_sram.c code
File name: ipmc-core/boot/boot.bin  size: 18072
Checksum section offset: 0002A0
Unused firmware flash: 86896
Unused boot loader flash: 6504
Making nvm blocks  
ipmc-core/util/mkhpm1img.c: In function hpm1_create_upgrade_image:
ipmc-core/util/mkhpm1img.c:359:5: warning: strncpy output truncated before terminating nul copying 8 bytes from a string of the same length [-Wstringop-truncation]
     strncpy((char *)hpm1_image.file_header.signature
     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  HPM1_IMAGE_SIGNATURE
  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
HPM.1 Image Creation Utility. Pentair Technical Products
Creating HPM.1 Upgrade Image 'hpm1fw.img'.
Processing file 'ipmc-core/hal/firmware.bin'.
HPM.1 Image Creation Utility. Pentair Technical Products
Creating HPM.1 Upgrade Image 'hpm1boot.img'.
Processing file 'ipmc-core/boot/boot.bin'.
HPM.1 Image Creation Utility. Pentair Technical Products
Creating HPM.1 Upgrade Image 'hpm1fru.img'.
Processing file 'ipmc-config/fru-info.bin'.
File name: ipmc-core/iap/iap.bin  size: 478895
Checksum section offset: 000004
HPM.1 Image Creation Utility. Pentair Technical Products
Creating HPM.1 Upgrade Image 'hpm1iap.img'.
Processing file 'ipmc-core/iap/iap.bin'.
HPM.1 Image Creation Utility. Pentair Technical Products
Creating HPM.1 Upgrade Image 'hpm1all.img'.
--> Compilation successfully finished
--> Archive creation successfully finished
--> Process finished
                   

--> Binaries: https://cern-ipmc.web.cern.ch/downloadbin/Und0y2uCln
--> Download and extract hpm.1 image
5

Hi Mathieu,

Right, it needed the change in the decode call in your compile.py script:

                strVar = strVar + chunk.decode("utf-8", "ignore")

Glad to hear. It does work now for you. Cheers,
Ralf.