[Linaro-validation] LAVA TEE

[Albarran, Josue]

Hi,

Has anyone tried to set up a LAVA instance where another test management system invokes this instance just for test execution and return the results to the other test system?

In other words, using LAVA as a Test Execution Engine. Any pointers?

Thanks,

Josue

[Neil Williams]

On Fri, 12 Aug 2016 14:52:11 +0000

Post by Albarran, Josue
Hi,
Has anyone tried to set up a LAVA instance where another test
management system invokes this instance just for test execution and
return the results to the other test system?
In other words, using LAVA as a Test Execution Engine. Any pointers?

That's what kernelci.org does - it's what any frontend for LAVA would
do, along with a tool to build things to be tested, e.g. Jenkins or
Travis. In a lot of cases, such build tools run tests as well,
especially when the build tests don't care about what hardware is used
to run the build. LAVA comes in where the hardware upon which those
tests run is fundamental to whether the tests pass or fail.

A build system is triggered by hooks in git and the scripts use the
LAVA API to submit those for test execution. The same API is then used
to retrieve the results and present the results with additional
functionality.

LAVA knows nothing about the tested product (the Linux source code
tree in the case of kernelci) beyond what is needed to automate the
test.

The kernelci.org frontend allows the results to be viewed in ways which
are independent of LAVA. (kernelci uses multiple LAVA labs and one lab
which does not run LAVA.) Each lab is independent and supports jobs
from users other than the kernelci scripts.

It's a question of perspective and terminology. Please describe,
carefully, what you are actually trying to achieve and at what points
specific physical hardware is required to be able to run certain tests.

[Albarran, Josue]

Hi,

Thanks for the reply.

Basically, let's say we have a system that submits tests and works as the LAVA server. This system does all the logic and it has a test master controller that works as a dispatcher to different test execution engines (TEEs). This controller has a TEE adapter that sends an XML file with all the details of the test job. I want to be able to convert this XML to YAML and use LAVA to execute the test plan and report results back to the server using the same TEE adapter.

I shouldn't need to install the entire server, correct? Can I install just the LAVA v2 dispatcher for this purpose? Can I still use lava-dispatch command or something similar in LAVA v2?

Thanks again.

-----Original Message-----
From: Neil Williams [mailto:***@debian.org]
Sent: Friday, August 12, 2016 12:14 PM
To: Albarran, Josue
Cc: linaro-***@lists.linaro.org
Subject: Re: [Linaro-validation] LAVA TEE

On Fri, 12 Aug 2016 14:52:11 +0000

Post by Albarran, Josue
Hi,
Has anyone tried to set up a LAVA instance where another test
management system invokes this instance just for test execution and
return the results to the other test system?
In other words, using LAVA as a Test Execution Engine. Any pointers?

That's what kernelci.org does - it's what any frontend for LAVA would do, along with a tool to build things to be tested, e.g. Jenkins or Travis. In a lot of cases, such build tools run tests as well, especially when the build tests don't care about what hardware is used to run the build. LAVA comes in where the hardware upon which those tests run is fundamental to whether the tests pass or fail.

A build system is triggered by hooks in git and the scripts use the LAVA API to submit those for test execution. The same API is then used to retrieve the results and present the results with additional functionality.

LAVA knows nothing about the tested product (the Linux source code tree in the case of kernelci) beyond what is needed to automate the test.

The kernelci.org frontend allows the results to be viewed in ways which are independent of LAVA. (kernelci uses multiple LAVA labs and one lab which does not run LAVA.) Each lab is independent and supports jobs from users other than the kernelci scripts.

It's a question of perspective and terminology. Please describe, carefully, what you are actually trying to achieve and at what points specific physical hardware is required to be able to run certain tests.

[Albarran, Josue]

Hi,

This is a follow-up message on the last message I sent. I will appreciate a response.

Thanks,

Josue

-----Original Message-----
From: Albarran, Josue
Sent: Friday, August 12, 2016 2:16 PM
To: 'Neil Williams'
Cc: linaro-***@lists.linaro.org
Subject: RE: [Linaro-validation] LAVA TEE

Hi,

Thanks for the reply.

Basically, let's say we have a system that submits tests and works as the LAVA server. This system does all the logic and it has a test master controller that works as a dispatcher to different test execution engines (TEEs). This controller has a TEE adapter that sends an XML file with all the details of the test job. I want to be able to convert this XML to YAML and use LAVA to execute the test plan and report results back to the server using the same TEE adapter.

I shouldn't need to install the entire server, correct? Can I install just the LAVA v2 dispatcher for this purpose? Can I still use lava-dispatch command or something similar in LAVA v2?

Thanks again.

-----Original Message-----
From: Neil Williams [mailto:***@debian.org]
Sent: Friday, August 12, 2016 12:14 PM
To: Albarran, Josue
Cc: linaro-***@lists.linaro.org
Subject: Re: [Linaro-validation] LAVA TEE

On Fri, 12 Aug 2016 14:52:11 +0000

Post by Albarran, Josue
Hi,
Has anyone tried to set up a LAVA instance where another test
management system invokes this instance just for test execution and
return the results to the other test system?
In other words, using LAVA as a Test Execution Engine. Any pointers?

That's what kernelci.org does - it's what any frontend for LAVA would do, along with a tool to build things to be tested, e.g. Jenkins or Travis. In a lot of cases, such build tools run tests as well, especially when the build tests don't care about what hardware is used to run the build. LAVA comes in where the hardware upon which those tests run is fundamental to whether the tests pass or fail.

A build system is triggered by hooks in git and the scripts use the LAVA API to submit those for test execution. The same API is then used to retrieve the results and present the results with additional functionality.

LAVA knows nothing about the tested product (the Linux source code tree in the case of kernelci) beyond what is needed to automate the test.

The kernelci.org frontend allows the results to be viewed in ways which are independent of LAVA. (kernelci uses multiple LAVA labs and one lab which does not run LAVA.) Each lab is independent and supports jobs from users other than the kernelci scripts.

It's a question of perspective and terminology. Please describe, carefully, what you are actually trying to achieve and at what points specific physical hardware is required to be able to run certain tests.