[Linaro-validation] Pipeline Job Submission

[Albarran, Josue]

Hi,

I'm having an issue where the board (beaglebone-black) starts the booting process using the u-boot commands that I specify but gets interrupted before it gets to the "am335x-evm login:" login prompt. I get the following exception http://pastebin.ubuntu.com/19177228/. Any known fix for this issue?

This is the complete boot log section of the pipeline actions http://pastebin.ubuntu.com/19177444/. Also, my YAML job definition can be found here http://pastebin.ubuntu.com/19177546/.

I will appreciate a response.

Thanks,

Josue

[Matt Hart]

Have you tried this boot outside of Lava?

It's either a timeout is too short and doesn't give your board enough
time to get to the login prompt, or the rootfs you are loading does
not produce a login shell.

Hi,
I’m having an issue where the board (beaglebone-black) starts the booting
process using the u-boot commands that I specify but gets interrupted before
it gets to the “am335x-evm login:” login prompt. I get the following
exception http://pastebin.ubuntu.com/19177228/. Any known fix for this
issue?
This is the complete boot log section of the pipeline actions
http://pastebin.ubuntu.com/19177444/. Also, my YAML job definition can be
found here http://pastebin.ubuntu.com/19177546/.
I will appreciate a response.
Thanks,
Josue
_______________________________________________
linaro-validation mailing list
https://lists.linaro.org/mailman/listinfo/linaro-validation

[Albarran, Josue]

Yes, I have tried it outside and it produces the login shell. I was thinking the same for the timeout. How can I change this timeout in order to give the board sufficient time to get to the login prompt? I even made the timeout for the boot action to 6 minutes and still have the same issue.

-----Original Message-----
From: Matt Hart [mailto:***@linaro.org]
Sent: Tuesday, July 12, 2016 9:29 AM
To: Albarran, Josue
Cc: linaro-***@lists.linaro.org
Subject: Re: [Linaro-validation] Pipeline Job Submission

Have you tried this boot outside of Lava?

It's either a timeout is too short and doesn't give your board enough time to get to the login prompt, or the rootfs you are loading does not produce a login shell.

Hi,
I’m having an issue where the board (beaglebone-black) starts the
booting process using the u-boot commands that I specify but gets
interrupted before it gets to the “am335x-evm login:” login prompt. I
get the following exception http://pastebin.ubuntu.com/19177228/. Any
known fix for this issue?
This is the complete boot log section of the pipeline actions
http://pastebin.ubuntu.com/19177444/. Also, my YAML job definition can
be found here http://pastebin.ubuntu.com/19177546/.
I will appreciate a response.
Thanks,
Josue
_______________________________________________
linaro-validation mailing list
https://lists.linaro.org/mailman/listinfo/linaro-validation

[Neil Williams]

On Tue, 12 Jul 2016 14:32:52 +0000

Post by Albarran, Josue
Yes, I have tried it outside and it produces the login shell. I was
thinking the same for the timeout. How can I change this timeout in
order to give the board sufficient time to get to the login prompt? I
even made the timeout for the boot action to 6 minutes and still have
the same issue.

I don't think it is a timeout - you've got 360s in that timeout.
start: 2.4.5 auto-login-action (max 360s)

The standard job for this device only needs:
start: 2.4.5 auto-login-action (max 120s)
https://staging.validation.linaro.org/scheduler/job/152266#action_2-4-5

The timeout itself is set in the boot action - 2 minutes in the case of
the standard job.

We created standard jobs for just this reason - to reduce the number of
variables.

https://staging.validation.linaro.org/scheduler/job/152266

This is known to work and gives you a base point to compare with your
job. The metadata links to how the files were built.

Staging is using the new code scheduled for the 2016.7 release, so the
logs look different.

Your problem looks like a misunderstanding of the prompts list:

prompts:
- "am335x-evm login:"
method: u-boot

That is not the prompt of the eventual test shell, that is the
auto-login prompt:

auto_login:
login_prompt: 'login:'
username: root
prompts:
- '***@jessie:'

The am335x-evm is a bad idea in this case. The auto-login prompt needs
to be independent of the hostname of the device or it will break as
soon as you add a second device or even a second image.

You need to know what prompt this image will give in a normal boot,
then set that in the prompts list - it is NOT likely to be login:

If your system needs a password as well as a user, you'll need to
specify the password_prompt and password in the auto_login section.

What is happening is not that the test times out, it is that you've
asked the test job to do something which is actually different to what
you do on the command line, so with a lack of response, it will timeout
no matter how long you wait.

Post by Albarran, Josue
-----Original Message-----
Sent: Tuesday, July 12, 2016 9:29 AM
To: Albarran, Josue
Subject: Re: [Linaro-validation] Pipeline Job Submission
Have you tried this boot outside of Lava?
It's either a timeout is too short and doesn't give your board enough
time to get to the login prompt, or the rootfs you are loading does
not produce a login shell.

Hi,
I’m having an issue where the board (beaglebone-black) starts the
booting process using the u-boot commands that I specify but gets
interrupted before it gets to the “am335x-evm login:” login prompt.
I get the following exception http://pastebin.ubuntu.com/19177228/.
Any known fix for this issue?
This is the complete boot log section of the pipeline actions
http://pastebin.ubuntu.com/19177444/. Also, my YAML job definition
can be found here http://pastebin.ubuntu.com/19177546/.
I will appreciate a response.
Thanks,
Josue
_______________________________________________
linaro-validation mailing list
https://lists.linaro.org/mailman/listinfo/linaro-validation

_______________________________________________
linaro-validation mailing list
https://lists.linaro.org/mailman/listinfo/linaro-validation

[Matt Hart]

I've also noticed on this job YAML: http://pastebin.ubuntu.com/19177546/

You need to indent compression underneath nfsrootfs, so it would be

nfsrootfs:
url: file:///home/jalbarran/ti-filesystem/tisdk-rootfs-image-am335x-evm.tar.gz
compression: gz

because you are declaring the compression of the nfsrootfs.

Post by Neil Williams
On Tue, 12 Jul 2016 14:32:52 +0000

Post by Albarran, Josue
Yes, I have tried it outside and it produces the login shell. I was
thinking the same for the timeout. How can I change this timeout in
order to give the board sufficient time to get to the login prompt? I
even made the timeout for the boot action to 6 minutes and still have
the same issue.

I don't think it is a timeout - you've got 360s in that timeout.
start: 2.4.5 auto-login-action (max 360s)
start: 2.4.5 auto-login-action (max 120s)
https://staging.validation.linaro.org/scheduler/job/152266#action_2-4-5
The timeout itself is set in the boot action - 2 minutes in the case of
the standard job.
We created standard jobs for just this reason - to reduce the number of
variables.
https://staging.validation.linaro.org/scheduler/job/152266
This is known to work and gives you a base point to compare with your
job. The metadata links to how the files were built.
Staging is using the new code scheduled for the 2016.7 release, so the
logs look different.
- "am335x-evm login:"
method: u-boot
That is not the prompt of the eventual test shell, that is the
login_prompt: 'login:'
username: root
The am335x-evm is a bad idea in this case. The auto-login prompt needs
to be independent of the hostname of the device or it will break as
soon as you add a second device or even a second image.
You need to know what prompt this image will give in a normal boot,
If your system needs a password as well as a user, you'll need to
specify the password_prompt and password in the auto_login section.
What is happening is not that the test times out, it is that you've
asked the test job to do something which is actually different to what
you do on the command line, so with a lack of response, it will timeout
no matter how long you wait.

Post by Albarran, Josue
-----Original Message-----
Sent: Tuesday, July 12, 2016 9:29 AM
To: Albarran, Josue
Subject: Re: [Linaro-validation] Pipeline Job Submission
Have you tried this boot outside of Lava?
It's either a timeout is too short and doesn't give your board enough
time to get to the login prompt, or the rootfs you are loading does
not produce a login shell.

Hi,
I’m having an issue where the board (beaglebone-black) starts the
booting process using the u-boot commands that I specify but gets
interrupted before it gets to the “am335x-evm login:” login prompt.
I get the following exception http://pastebin.ubuntu.com/19177228/.
Any known fix for this issue?
This is the complete boot log section of the pipeline actions
http://pastebin.ubuntu.com/19177444/. Also, my YAML job definition
can be found here http://pastebin.ubuntu.com/19177546/.
I will appreciate a response.
Thanks,
Josue
_______________________________________________
linaro-validation mailing list
https://lists.linaro.org/mailman/listinfo/linaro-validation

_______________________________________________
linaro-validation mailing list
https://lists.linaro.org/mailman/listinfo/linaro-validation

--
Neil Williams
=============
http://www.linux.codehelp.co.uk/
_______________________________________________
linaro-validation mailing list
https://lists.linaro.org/mailman/listinfo/linaro-validation

[Albarran, Josue]

Hi,

Thanks for your help, I fixed it and now it works.

I want to be able to run a script, have it return a value and have LAVA detect pass or fail based on the output of the script. Would something like this http://pastebin.ubuntu.com/19277125/ work inside of the test action of the job definition? If so, where should the script be located in order for lava to find it? I read that currently there is only support for Git and Inline, is this correct?

This will help me understand the process so I can run functional LTP tests in the future.

I will appreciate your help.

Thanks,

Josue

-----Original Message-----
From: Neil Williams [mailto:***@debian.org]
Sent: Tuesday, July 12, 2016 2:08 PM
To: Albarran, Josue
Cc: linaro-***@lists.linaro.org
Subject: Re: [Linaro-validation] Pipeline Job Submission

On Tue, 12 Jul 2016 14:32:52 +0000

Post by Albarran, Josue
Yes, I have tried it outside and it produces the login shell. I was
thinking the same for the timeout. How can I change this timeout in
order to give the board sufficient time to get to the login prompt? I
even made the timeout for the boot action to 6 minutes and still have
the same issue.

I don't think it is a timeout - you've got 360s in that timeout.
start: 2.4.5 auto-login-action (max 360s)

The standard job for this device only needs:
start: 2.4.5 auto-login-action (max 120s)
https://staging.validation.linaro.org/scheduler/job/152266#action_2-4-5

The timeout itself is set in the boot action - 2 minutes in the case of the standard job.

We created standard jobs for just this reason - to reduce the number of variables.

https://staging.validation.linaro.org/scheduler/job/152266

This is known to work and gives you a base point to compare with your job. The metadata links to how the files were built.

Staging is using the new code scheduled for the 2016.7 release, so the logs look different.

Your problem looks like a misunderstanding of the prompts list:

prompts:
- "am335x-evm login:"
method: u-boot

That is not the prompt of the eventual test shell, that is the auto-login prompt:

auto_login:
login_prompt: 'login:'
username: root
prompts:
- '***@jessie:'

The am335x-evm is a bad idea in this case. The auto-login prompt needs to be independent of the hostname of the device or it will break as soon as you add a second device or even a second image.

You need to know what prompt this image will give in a normal boot, then set that in the prompts list - it is NOT likely to be login:

If your system needs a password as well as a user, you'll need to specify the password_prompt and password in the auto_login section.

What is happening is not that the test times out, it is that you've asked the test job to do something which is actually different to what you do on the command line, so with a lack of response, it will timeout no matter how long you wait.

Post by Albarran, Josue
-----Original Message-----
Sent: Tuesday, July 12, 2016 9:29 AM
To: Albarran, Josue
Subject: Re: [Linaro-validation] Pipeline Job Submission
Have you tried this boot outside of Lava?
It's either a timeout is too short and doesn't give your board enough
time to get to the login prompt, or the rootfs you are loading does
not produce a login shell.

Hi,
I’m having an issue where the board (beaglebone-black) starts the
booting process using the u-boot commands that I specify but gets
interrupted before it gets to the “am335x-evm login:” login prompt.
I get the following exception http://pastebin.ubuntu.com/19177228/.
Any known fix for this issue?
This is the complete boot log section of the pipeline actions
http://pastebin.ubuntu.com/19177444/. Also, my YAML job definition
can be found here http://pastebin.ubuntu.com/19177546/.
I will appreciate a response.
Thanks,
Josue
_______________________________________________
linaro-validation mailing list
https://lists.linaro.org/mailman/listinfo/linaro-validation

_______________________________________________
linaro-validation mailing list
https://lists.linaro.org/mailman/listinfo/linaro-validation

--


Neil Williams
=============
http://www.linux.codehelp.co.uk/

[Neil Williams]

On Wed, 13 Jul 2016 14:47:10 +0000

Post by Albarran, Josue
Hi,
Thanks for your help, I fixed it and now it works.
I want to be able to run a script, have it return a value and have
LAVA detect pass or fail based on the output of the script.

Depends on the output - the simplest way to set a pass or fail is to
call lava-test-case directly and you can do that inside your script.
(It's added to the PATH inside the test job.)

Parsing patterns against script output gets difficult and hard to
debug. You can do the parsing inside the script and call lava-test-case
too:

https://git.linaro.org/people/neil.williams/temp-functional-tests.git/blob/HEAD:/lava/dispatcher-branch-unit-tests.yaml
calls
https://git.linaro.org/people/neil.williams/temp-functional-tests.git/blob/HEAD:/lava/unittests.sh
which does some parsing and then calls lava-test-case.

This way, you can double-check your scripts against the output in the
log files created within LAVA.

Post by Albarran, Josue
Would
something like this http://pastebin.ubuntu.com/19277125/ work inside
of the test action of the job definition?

The subshell is entirely unnecessary, just use:.

run:
steps:
- ./my-script.sh arguments

To put that into the test job definition - the part you submit to LAVA
- so it needs to be inline and you'd need a previous step which
downloads or installs that script.

run:
deps:
- wget
steps:
- wget http://example.com/my-script.sh
- chmod 755 ./my-script.sh
- ./my-script.sh arguments

There are limitations on how complex these run steps can be, so a
script is generally better than trying to combine shell commands with
pipes and redirects.

Inline is useful in development and will be recommended when using the
synchronisation within multinode but a VCS like git is the best place
for the actual test definitions.

To ask LAVA to fetch & execute the test definition from git use
something like:

- test:
timeout:
minutes: 1
definitions:
- repository: git://git.linaro.org/qa/test-definitions.git
from: git
path: ubuntu/smoke-tests-basic.yaml
name: smoke-tests

Distinguish between the job definition and the test definition. Inline
tests live in the job definition but are limited compared to tests
which live in the test definition. Test definitions benefit greatly
from being in a VCS. For "important" tests, all of your results should
come from a test definition hosted in a git repository.

Post by Albarran, Josue
If so, where should the
script be located in order for lava to find it?

Use a git repo which can be cloned for you, then the script lives in the
top level directory of the git repo or a path below that directory.

Post by Albarran, Josue
I read that currently
there is only support for Git and Inline, is this correct?

There is bzr support but few people use it - git is the version
control system to use. It is best to use a version control system so
that you can track what changed in the test definitions themselves. V2
records the commit ID hash of the clone so that you can see exactly
which version of the test definition was used.

Post by Albarran, Josue
This will help me understand the process so I can run functional LTP tests in the future.

There already are LTP definitions running for certain devices in LAVA.

https://git.linaro.org/qa/test-definitions.git/blob/HEAD:/ubuntu/ltp.yaml

That uses a git repo to provide a file in ./common/scripts/ which does
most of the work but does rely on a parsing pattern.

[Albarran, Josue]

Hi,

Just to be clear, lava-test-case are predefined test cases that lie within lava? If so, where can I find these?

It has two forms, the one that describes the outcome of the test case and the other that takes the shell command to run. Like the following:

steps:
- "lava-test-case simpletestcase --result pass"
- "lava-test-case fail-test --shell false"

Is simpletestcase and fail-test a predefined test or just a random name?

Thanks,

Josue

-----Original Message-----
From: Neil Williams [mailto:***@debian.org]
Sent: Wednesday, July 13, 2016 10:17 AM
To: Albarran, Josue
Cc: linaro-***@lists.linaro.org
Subject: Re: [Linaro-validation] Pipeline Job Submission

On Wed, 13 Jul 2016 14:47:10 +0000

Post by Albarran, Josue
Hi,
Thanks for your help, I fixed it and now it works.
I want to be able to run a script, have it return a value and have
LAVA detect pass or fail based on the output of the script.

Depends on the output - the simplest way to set a pass or fail is to call lava-test-case directly and you can do that inside your script.
(It's added to the PATH inside the test job.)

Parsing patterns against script output gets difficult and hard to debug. You can do the parsing inside the script and call lava-test-case
too:

https://git.linaro.org/people/neil.williams/temp-functional-tests.git/blob/HEAD:/lava/dispatcher-branch-unit-tests.yaml
calls
https://git.linaro.org/people/neil.williams/temp-functional-tests.git/blob/HEAD:/lava/unittests.sh
which does some parsing and then calls lava-test-case.

This way, you can double-check your scripts against the output in the log files created within LAVA.

Post by Albarran, Josue
Would
something like this http://pastebin.ubuntu.com/19277125/ work inside
of the test action of the job definition?

The subshell is entirely unnecessary, just use:.

run:
steps:
- ./my-script.sh arguments

To put that into the test job definition - the part you submit to LAVA
- so it needs to be inline and you'd need a previous step which downloads or installs that script.

run:
deps:
- wget
steps:
- wget http://example.com/my-script.sh
- chmod 755 ./my-script.sh
- ./my-script.sh arguments

There are limitations on how complex these run steps can be, so a script is generally better than trying to combine shell commands with pipes and redirects.

Inline is useful in development and will be recommended when using the synchronisation within multinode but a VCS like git is the best place for the actual test definitions.

To ask LAVA to fetch & execute the test definition from git use something like:

- test:
timeout:
minutes: 1
definitions:
- repository: git://git.linaro.org/qa/test-definitions.git
from: git
path: ubuntu/smoke-tests-basic.yaml
name: smoke-tests

Distinguish between the job definition and the test definition. Inline tests live in the job definition but are limited compared to tests which live in the test definition. Test definitions benefit greatly from being in a VCS. For "important" tests, all of your results should come from a test definition hosted in a git repository.

Post by Albarran, Josue
If so, where should the
script be located in order for lava to find it?

Use a git repo which can be cloned for you, then the script lives in the top level directory of the git repo or a path below that directory.

Post by Albarran, Josue
I read that currently
there is only support for Git and Inline, is this correct?

There is bzr support but few people use it - git is the version control system to use. It is best to use a version control system so that you can track what changed in the test definitions themselves. V2 records the commit ID hash of the clone so that you can see exactly which version of the test definition was used.

Post by Albarran, Josue
This will help me understand the process so I can run functional LTP tests in the future.

There already are LTP definitions running for certain devices in LAVA.

https://git.linaro.org/qa/test-definitions.git/blob/HEAD:/ubuntu/ltp.yaml

That uses a git repo to provide a file in ./common/scripts/ which does most of the work but does rely on a parsing pattern.

[Neil Williams]

On Fri, 15 Jul 2016 15:02:12 +0000

Post by Albarran, Josue
Hi,
Just to be clear, lava-test-case are predefined test cases that lie
within lava?

No. lava-test-case is a small shell script that can be run inside a
lava test job to register a test case result as pass or fail and to
record measurements with units.

Post by Albarran, Josue
If so, where can I find these?

LAVA has no predefined test cases, just some scripts to record tests
that happen inside the job. You define what is to be tested. There are
examples of test definitions created by others - you can see any number
of those by looking at jobs that other people have run within LAVA.

Post by Albarran, Josue
It has two forms, the one that describes the outcome of the test case
- "lava-test-case simpletestcase --result pass"
- "lava-test-case fail-test --shell false"
Is simpletestcase and fail-test a predefined test or just a random name?

User specified name, a label that applies to this individual test case.

The --shell option takes a command but complex commands using pipes and
redirects need to be done using scripts which you create alongside the
Lava Test Shell Definition in the git repo specified in the test job.

Post by Albarran, Josue
Thanks,
Josue
-----Original Message-----
Sent: Wednesday, July 13, 2016 10:17 AM
To: Albarran, Josue
Subject: Re: [Linaro-validation] Pipeline Job Submission
On Wed, 13 Jul 2016 14:47:10 +0000

Post by Albarran, Josue
Hi,
Thanks for your help, I fixed it and now it works.
I want to be able to run a script, have it return a value and have
LAVA detect pass or fail based on the output of the script.

Depends on the output - the simplest way to set a pass or fail is to
call lava-test-case directly and you can do that inside your script.
(It's added to the PATH inside the test job.)
Parsing patterns against script output gets difficult and hard to
debug. You can do the parsing inside the script and call
https://git.linaro.org/people/neil.williams/temp-functional-tests.git/blob/HEAD:/lava/dispatcher-branch-unit-tests.yaml
calls
https://git.linaro.org/people/neil.williams/temp-functional-tests.git/blob/HEAD:/lava/unittests.sh
which does some parsing and then calls lava-test-case.
This way, you can double-check your scripts against the output in the
log files created within LAVA.

Post by Albarran, Josue
Would
something like this http://pastebin.ubuntu.com/19277125/ work
inside of the test action of the job definition?

The subshell is entirely unnecessary, just use:.
- ./my-script.sh arguments
To put that into the test job definition - the part you submit to LAVA
- so it needs to be inline and you'd need a previous step which
downloads or installs that script.
- wget
- wget http://example.com/my-script.sh
- chmod 755 ./my-script.sh
- ./my-script.sh arguments
There are limitations on how complex these run steps can be, so a
script is generally better than trying to combine shell commands with
pipes and redirects.
Inline is useful in development and will be recommended when using
the synchronisation within multinode but a VCS like git is the best
place for the actual test definitions.
minutes: 1
- repository: git://git.linaro.org/qa/test-definitions.git
from: git
path: ubuntu/smoke-tests-basic.yaml
name: smoke-tests
Distinguish between the job definition and the test definition.
Inline tests live in the job definition but are limited compared to
tests which live in the test definition. Test definitions benefit
greatly from being in a VCS. For "important" tests, all of your
results should come from a test definition hosted in a git repository.

Post by Albarran, Josue
If so, where should the
script be located in order for lava to find it?

Use a git repo which can be cloned for you, then the script lives in
the top level directory of the git repo or a path below that
directory.

Post by Albarran, Josue
I read that currently
there is only support for Git and Inline, is this correct?

There is bzr support but few people use it - git is the version
control system to use. It is best to use a version control system so
that you can track what changed in the test definitions themselves.
V2 records the commit ID hash of the clone so that you can see
exactly which version of the test definition was used.

Post by Albarran, Josue
This will help me understand the process so I can run functional
LTP tests in the future.

There already are LTP definitions running for certain devices in LAVA.
https://git.linaro.org/qa/test-definitions.git/blob/HEAD:/ubuntu/ltp.yaml
That uses a git repo to provide a file in ./common/scripts/ which
does most of the work but does rely on a parsing pattern.
--
Neil Williams
=============
http://www.linux.codehelp.co.uk/