Commit Graph

346 Commits

Author SHA1 Message Date
Andrey Melnikov
83a4238153 Merge pull request #617 from Vafilor/feat/workspace.listing
feat: workspace list filtering, sorting and statistics
2020-09-29 11:20:46 -07:00
Andrey Melnikov
77716ba56b chore: updated method docs 2020-09-29 11:17:50 -07:00
Andrey Melnikov
8b4a70d958 feat: added support for sorting and filtering workspaces.
* Also added a LabelFilter interface
2020-09-28 12:09:16 -07:00
Rush Tehrani
91b97d9243 Merge pull request #611 from aleksandrmelnikov/feat/core.607-revert.to.resource.limits.requests
feat: Undo pod anti-affinity for scheduling nodes, use resource requests instead.
2020-09-25 14:02:02 -07:00
Aleksandr Melnikov
e7cef240c4 Fixing code that assigns a GPU limit for a workflow.
- This GPU assignment works for cron created workflows as well.
2020-09-25 10:38:21 -07:00
Aleksandr Melnikov
dca6db842c Adding back injectContainerResourceQuotas for workflows.
- This time, the node capacity is grabbed from running nodes instead
of information from configmap (which is grabbed from params.yaml)
- Added support for two different instance-type keys.
Note that GPU support does not work at this time.
- There is no ResourceName "nvidia.com/gpu" in library code, so it throws
a deref nil error.
2020-09-24 17:28:54 -07:00
Andrey Melnikov
f7770618ca update: added parameter to control listing of system workflows 2020-09-24 12:32:35 -07:00
Aleksandr Melnikov
77812419d2 Removing pod anti-affinity injection from workflows.
- Workspaces end up calling workflows. This has to be removed
as part of switching back to resource requests.
2020-09-24 11:03:37 -07:00
Andrey Melnikov
6fa123b122 feat: added explicit nulls sorting for workflow execution columns. Idea is to treat null as an empty/zero value. So in asc, it is first. 2020-09-23 19:13:53 -07:00
Andrey Melnikov
b1d0ab1d59 update: added workflow template name and uid to select of workflow executions 2020-09-23 16:20:38 -07:00
Rush Tehrani
c45231c106 Merge pull request #585 from aleksandrmelnikov/feat/core.583-workflows.use.pod.anti.affinity
feat: Updated Workflows, and Cron created Workflows, to ensure they get their own pod by using Pod AntiAffinity. This replaces resource limits and/or requests.
2020-09-21 16:20:13 -07:00
Aleksandr Melnikov
9ec45e4f34 Formatting feedback and fix. 2020-09-21 16:12:53 -07:00
Aleksandr Melnikov
48e2050e97 Adjusting code so that we don't need the nodePoolLabel to figure
out the selected Node.
- The nodePoolLabel can change with the params.yaml and different
k8s versions.
2020-09-18 16:34:07 -07:00
Aleksandr Melnikov
d42f88e04c Adding flag to decide when to add pod affinity.
- This avoids adding it every time the loop finds a nodeSelector that's
not nil in a template
2020-09-18 15:53:00 -07:00
Aleksandr Melnikov
6dd7c0ac70 Adjusting the pod anti-affinity per feedback.
- The pod anti-affinity should be set for a template that has a nodeSelector
value (not nil).
- If the template does not have a nodeSelector, we do nothing.
2020-09-18 15:51:48 -07:00
Aleksandr Melnikov
31076bc70d Adding systemConfig to function.
- Also added error return
2020-09-18 15:50:28 -07:00
Aleksandr Melnikov
aaf20b4ab6 Updating code to use the returned wf. 2020-09-17 17:43:40 -07:00
Aleksandr Melnikov
7bc1056bc6 Updating function to return the updated workflow. 2020-09-17 17:42:23 -07:00
Aleksandr Melnikov
880e8ba082 Removing prior code that injected resource requests and limits
to workflows.
2020-09-17 17:08:06 -07:00
Aleksandr Melnikov
ec634b66ca Adding function to ensure a workflow gets a dedicated node for all the
templates it executes.
- Note that workflows executed by cron are also affected.
2020-09-17 17:06:20 -07:00
Andrey Melnikov
d524c3cb66 update: fixed issue where GetWorkflowExecutionStatisticsForNamespace included workspace workflows (is_system = true). Added support for filtering by phase. 2020-09-16 19:37:10 -07:00
Andrey Melnikov
a9953683a9 feat: added endpoint to get the workflow execution statistics for a namespace 2020-09-16 19:37:10 -07:00
Andrey Melnikov
d4d4884e5b feat: initial pagination updates 2020-09-16 18:26:51 -07:00
Aleksandr Melnikov
029469e031 Changing the secret "key" value.
- Otherwise, the workflow can't get access to GCS bucket.
2020-09-08 11:10:05 -07:00
Andrey Melnikov
5d3345ded2 update: removed port since default is 80 2020-08-17 16:17:10 -07:00
Andrey Melnikov
0274785f72 fix: removed port environment variable as it is no longer needed since host is not using an environment variable 2020-08-17 16:14:30 -07:00
Andrey Melnikov
bcec3c13fd fix: update core pod host to use string instead of environment variable which evaluates to a static ip 2020-08-17 16:07:47 -07:00
rushtehrani
ce805bfa20 feat: Support script templates in Workflow 2020-08-12 22:31:42 -07:00
Andrey Melnikov
4b4cccbd74 Merge pull request #486 from onepanelio/feat/remove-nvidia-smi-mount
chore: Remove nvidia-smi mount
2020-08-11 10:10:57 -07:00
Aleksandr Melnikov
d41dad0074 Merge pull request #493 from onepanelio/feat/onepanelio.labels.upgrade
feat: upgrade label structure to support querying with multiple labels
2020-08-10 17:01:19 -07:00
Andrey Melnikov
8586639a6c update: cron workflows to use new labels 2020-08-09 16:15:05 -07:00
Andrey Melnikov
c81c2d7672 update: fixed issues where workspace template version labels were not being correctly set/got. Also updated generic server endpoints with new logic 2020-08-09 13:45:24 -07:00
Andrey Melnikov
f02e7791f7 update: updated labels for workflow templates and their versions 2020-08-08 15:56:17 -07:00
Andrey Melnikov
6edca5731b update: change reader to use bufio for buffered reading 2020-08-07 15:15:25 -07:00
Andrey Melnikov
63bdb69968 fix: issue where scanner could not handle long lines while reading logs 2020-08-07 15:10:55 -07:00
rushtehrani
b2e887c1c9 remove nvidia-smi mount 2020-08-06 14:18:21 -07:00
Andrey Melnikov
3e6a48ba1e fix: issue where running a workflow execution again failed. 2020-08-04 10:40:18 -07:00
Andrey Melnikov
aeaa447beb Revert "poc: Using a custom type for labels backed by JSONB"
This reverts commit 023713da32.
2020-08-03 15:26:08 -07:00
Andrey Melnikov
023713da32 poc: Using a custom type for labels backed by JSONB 2020-08-03 15:25:18 -07:00
Andrey Melnikov
7e63fa28d0 fix: issue where nil param values caused dereference errors 2020-07-29 22:27:53 -07:00
Andrey Melnikov
b0ec3194b6 fix: workflow execution uid = name now, we should not be returning the k8s uid. 2020-07-28 11:53:49 -07:00
Aleksandr Melnikov
16274dd946 Moving some attributes form S3Config to S3Provider. 2020-07-22 15:32:38 -07:00
Aleksandr Melnikov
fa96d6ef66 Merge branch 'dev' into feat/core.331-support.gcs.artifact.repository 2020-07-14 14:06:47 -07:00
Aleksandr Melnikov
6c0d3fe598 Refactored GCS client creation.
Added GetObject function to be consistent with S3 functionality.
Note
- pkg/client.go, line 88 passes in the JSON.
If ArtifactRepositoryGCSCondig was imported inside gcs.NewClient
code, an import cycle would be created.
2020-07-14 14:05:31 -07:00
Aleksandr Melnikov
e526505365 Fixing the Key to look for inside the secret. 2020-07-13 17:01:07 -07:00
Aleksandr Melnikov
993ce397aa Removing todo, verified Name is correct. 2020-07-13 16:02:14 -07:00
Aleksandr Melnikov
7a8df485ad Removing completed todos. 2020-07-10 17:53:33 -07:00
Aleksandr Melnikov
94ae9071aa Adding exit condition to for true when iterating through bucket files. 2020-07-10 17:19:03 -07:00
Aleksandr Melnikov
8e2b1ab106 Merge branch 'dev' into feat/core.331-support.gcs.artifact.repository 2020-07-09 18:09:50 -07:00
Aleksandr Melnikov
e83e5d495e Injecting GCS configuration into ArtifactRepository. 2020-07-09 18:02:18 -07:00