purgo build fail at linux

I try to build an app using purego on ubuntu but it fails on linux

2024-01-07T05:12:29.4981059Z # github.com/cyberhan123/stable-diffusion-desktop
2024-01-07T05:12:29.4983718Z dlopen: unhandled relocation for purego_dlopen (type 46 (SDYNIMPORT) rtype 7 (R_CALL))
2024-01-07T05:12:29.4989379Z dlsym: unhandled relocation for purego_dlsym (type 46 (SDYNIMPORT) rtype 7 (R_CALL))
2024-01-07T05:12:29.4991109Z dlerror: unhandled relocation for purego_dlerror (type 46 (SDYNIMPORT) rtype 7 (R_CALL))
2024-01-07T05:12:29.4992706Z dlclose: unhandled relocation for purego_dlclose (type 46 (SDYNIMPORT) rtype 7 (R_CALL))
2024-01-07T05:12:29.5058987Z   �[90m�[90m•�[0m�[0m �[39m�[39mCompiling application: �[0m�[0m�[30;101m�[30;101m  ERROR  �[0m�[0m �[91m�[91mexit status 1�[0m�[0m

The following is the complete github action log:

App, linuxamd64, ubuntu-latest.txt

@Cyberhan123

Hey @Cyberhan123 👋

Can you also provide the output of go env (feel free to redact any value that you don't feel comfortable sharing)?

In fact, there is in the log.

@TotallyGamerJet

DataDog/dd-trace-go#2504 (comment)

@TotallyGamerJet Do you have any insights?

I do not. Seems very strange since no changes to the dynamic linking have been made in quite a while

@Cyberhan123

@Cyberhan123 does it work if you set CGO_ENABLED=0?

If I am not mistaken, if relocations happens during build process it means the library is being statically linked. Now the question is: why would anyone want to static link libdl? I believe this could be caused by another package somewhere in the dependency tree that added -static or equivalent in the LDFLAGS directive for cgo.

Adding -x and -work flags should give us enough info to understand if it's the case by grepping in the files and logs. I should be able to try it myself tomorrow.

Relocations happen for any symbol that is in a different object file. Even if that symbol is one that is dynamically linked it must be relocated to the GOT & POT at build time.

However, I think you might be correct that there is something causing it to try and be statically linked which isn't possible with purego. Please do let me know what you find!

I was able to confirm on Ubunutu arm64 that building with #cgo LDFLAGS: -static produces the same output that is displayed above. This would suggest that this isn't an issue with purego as there is no way to statically link with purego since it requires libdl.so. Someone could rewrite the entire dynamic linker in Go but that sounds like it's begging to break.

@Cyberhan123

@Cyberhan123 does it work if you set CGO_ENABLED=0?

If it is pure go code, there is no problem,If cgo exists, the above error will be reported.

I was able to confirm on Ubunutu arm64 that building with #cgo LDFLAGS: -static produces the same output that is displayed above. This would suggest that this isn't an issue with purego as there is no way to statically link with purego since it requires libdl.so. Someone could rewrite the entire dynamic linker in Go but that sounds like it's begging to break.

I'm using wails to build the desktop app ,I need to investigate it, thanks for the heads up.

My code below also causes this problem:

CGO_ENABLED=1 go build main.go

package main

import (
	"fmt"
	"image"
	"image/color"
	"image/png"
	"os"

	sd "github.com/seasonjs/stable-diffusion"
)


// #include <stdio.h>
// #include <stdlib.h>
//
// void __lsan_do_leak_check(void);
//
//#cgo CFLAGS: -fsanitize=address
//#cgo LDFLAGS: -fsanitize=address
import "C"

func MemoryLeakCheck() error {
	diffusion, err := sd.NewCStableDiffusion("./deps/linux/lbsd-abi.so")
	if err != nil {
		return err
	}
	diffusion.SetLogCallBack(func(level sd.LogLevel, text string) {
		fmt.Printf("%s", text)
	})
	ctx := diffusion.NewCtx("./models/miniSD.ckpt", "", "", "", false, false, true, 4, sd.F16, sd.CUDA_RNG, sd.DEFAULT)
	defer diffusion.FreeCtx(ctx)

	images := diffusion.PredictImage(ctx, "british short hair cat, high quality", "", 0, 7.0, 128, 128, sd.EULER_A, 10, 43, 1)

	err = writeToFile(images[0].Data, int(images[0].Height), int(images[0].Width), "./assets/love_cat1.png")
	if err != nil {
		return err
	}
	C.__lsan_do_leak_check()
	return nil
}

func writeToFile(byteData []byte, height int, width int, outputPath string) error {

	img := image.NewRGBA(image.Rect(0, 0, width, height))

	for y := 0; y < height; y++ {
		for x := 0; x < width; x++ {
			idx := (y*width + x) * 3
			img.Set(x, y, color.RGBA{
				R: byteData[idx],
				G: byteData[idx+1],
				B: byteData[idx+2],
				A: 255,
			})
		}
	}

	file, err := os.Create(outputPath)
	if err != nil {
		return err
	}
	defer file.Close()

	err = png.Encode(file, img)
	if err != nil {
		return err
	}
	return nil
}

func main() {
	MemoryLeakCheck()
}

There is definitely something going on with the connection between having import "C" and import "github.com/seasonjs/stable-diffusion" because doing CGO_ENABLED=1 go build with the below code still produces the error. Is there another import "C" in stable-diffusion? I still don't think this is an issue fixable inside purego.

package main

import (
	_ "github.com/seasonjs/stable-diffusion"
)


import "C"


func main() {
}

No cgo code is used in stable-diffusion

@rsc

Let me make a small attempt: @rsc

It likely has to do with the fact that //go:cgo_import_dynamic is for internal linking while //go:cgo_import_static is for external linking. The former is what purego uses and the later can't be used outside the compiler. I just don't know why it worked before.

Fix issue 189 (#197)

I'll tag v0.5.2 and v0.6.0-alpha.4 a few days later, if we don't find any other issues.

Fix issue 189 (#197)

Let me reopen this. We are not able to cherry-pick this to 0.5 yet.

Fix issue 189 (#197)

reland: Fix issue 189 (#197)

Done

go.mod: upgrade purego v0.5.1 => upgrade purego v0.5.2 (#63)

Navigation Menu

Repository navigation

purgo build fail at linux #189

10 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

purgo build fail at linux #189

Description

Activity

eliottness commented on Jan 15, 2024

Cyberhan123 commented on Jan 15, 2024

hajimehoshi commented on Jan 15, 2024

TotallyGamerJet commented on Jan 15, 2024

TotallyGamerJet commented on Jan 15, 2024

eliottness commented on Jan 15, 2024

TotallyGamerJet commented on Jan 15, 2024

TotallyGamerJet commented on Jan 16, 2024

Cyberhan123 commented on Jan 16, 2024

Cyberhan123 commented on Jan 16, 2024

Cyberhan123 commented on Jan 16, 2024

TotallyGamerJet commented on Jan 16, 2024

Cyberhan123 commented on Jan 16, 2024

10 remaining items

Cyberhan123 commented on Jan 17, 2024

TotallyGamerJet commented on Jan 17, 2024

hajimehoshi commented on Jan 17, 2024

hajimehoshi commented on Jan 17, 2024

hajimehoshi commented on Jan 18, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions