All jobs are in `new` state and not running

[slava-vishnyakov]

Hi everybody!

I was trying to run jobs and everything was fine until I added bleve.Open into my script init.

The problem is that bleve.Open runs a file-based locking for concurrent processes.

Suddenly, jobs stopped working. All new Workers got into new state and that's about it.

After some debugging I found out that Worker works by starting second process and that process was basically locked, because bleve.Open was called before Admin initialization.

I think the documentation for qor/worker would benefit from information on how default cron runner works.

Something along these lines:

Worker runs by running second instance of your binary with --qor-job ID, so if you have any locks for concurrent processes - you need to have them after the Admin.MountTo(..) call

[slava-vishnyakov]

Otherwise qor is superb! Love it! Thanks for all hard work!

[mmt7172]

I am facing the same issue while using cli flags. OS is not able to create a new process for running worker job when I am using flag package of golang. I need to use it for getting environment specific variables. Can you please help me with a way around?

Thanks!

[dfang]

@mmt7172 use NewFlagSet,

https://github.com/qor/qor-example/blob/master/main.go#L36

I also have the same issue, no job are started. Any fixes for that ?

[jwldr]

@x0rzkov

as mentioned by slava-vishnyakov: workers run by spawning a second instance of your main binary with --qor-job <ID> as only argument passed to it

because of this several things can interfere with the qor/worker package leaving a job in new state (never actually running it) ...for example cmdline parsing packages like flaggy or go-arg and the order of instructions in your main func (especially when the program will not initialize admin if it is run with --qor-job <ID> )

When the binary runs with --qor-job <ID> ensure that it will always init admin with database connection and all required worker resources.

example:

package main

import (
	"fmt"
	"net/http"
	"time"

	goarg "github.com/alexflint/go-arg"
	"github.com/jinzhu/gorm"
	_ "github.com/jinzhu/gorm/dialects/sqlite"
	"github.com/qor/admin"
	"github.com/qor/worker"
)

// User struct
type User struct {
	gorm.Model
	Name string
}

// Product another GORM-backend model
type Product struct {
	gorm.Model
	Name        string
	Description string
}

func newWorker(admin *admin.Admin) *worker.Worker {
	// Define Worker
	w := worker.New()

	// Register Job
	w.RegisterJob(&worker.Job{
		Name: "Test",
		Handler: func(argument interface{}, qorJob worker.QorJobInterface) error {
			qorJob.AddLog("Started test...")

			for i := 1; i <= 100; i++ {
				time.Sleep(100 * time.Millisecond)
				qorJob.AddLog(fmt.Sprintf("doing somthing %v...", i))
				qorJob.SetProgress(uint(i))
			}
			qorJob.AddLog("Finished test")
			return nil
		},
		Resource: admin.NewResource(nil),
	})
	return w
}

func main() {

	var args struct {
		QorJob int `arg:"--qor-job"`
	}
	goarg.MustParse(&args)

	// Set up the database
	DB, _ := gorm.Open("sqlite3", "demo.db")
	DB.AutoMigrate(&User{}, &Product{})

	// Initalize
	Admin := admin.New(&admin.AdminConfig{DB: DB})

	// Create resources from GORM-backend model
	Admin.AddResource(&User{})
	Admin.AddResource(&Product{})
	Admin.AddResource(newWorker(Admin))

	if args.QorJob == 0 {

		// ..main app logic that is normally run if --qor-job is not present (0)

		// Initalize an HTTP request multiplexer
		mux := http.NewServeMux()

		// Mount admin to the mux
		Admin.MountTo("/admin", mux)

		fmt.Println("Listening on: 3000")
		http.ListenAndServe(":3000", mux)

	}
}

this example works only for jobs which will be started immediate (without worker.Schedule embedded in the job Resource) or if the schedule is set to something in the past, seeing the same behavior in the qor-example repo