Skip to content

Commit 2ddc413

Browse files
Canicula98Canicula98
authored
Merge pull request #63 from bsc-dom/6HxXam8Cs607MbbY
Pycompss_dummy
2 parents a8aa30b + faf98bc commit 2ddc413

330 files changed

Lines changed: 52174 additions & 4 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎examples/dds/.COMPSsWorker/57ea0678-a813-4399-b18f-b0bfaa5ac187/localhost/log/binding_worker.err‎

Whitespace-only changes.

‎examples/dds/.COMPSsWorker/57ea0678-a813-4399-b18f-b0bfaa5ac187/localhost/log/binding_worker.out‎

Whitespace-only changes.

‎examples/dds/.COMPSsWorker/57ea0678-a813-4399-b18f-b0bfaa5ac187/localhost/log/worker_localhost.err‎

Whitespace-only changes.

‎examples/dds/.COMPSsWorker/57ea0678-a813-4399-b18f-b0bfaa5ac187/localhost/log/worker_localhost.out‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
[(202)(2025-02-17 09:05:14,059) Worker] @<init> - No valid hostID provided to the tracing system. Provided ID: localhost
2+
[(204)(2025-02-17 09:05:14,061) Worker] @init - No streaming backend passed
3+
[(205)(2025-02-17 09:05:14,062) Worker] @init - No storage configuration file passed

‎examples/dds/cfg‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
{"working_dir":"/home/dcanocan/Desktop/Curro/git/dataclay/examples/dds","resources":"","project":""}

‎examples/dds/client.py‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
import sys
2+
print("Python executable:", sys.executable)
3+
print("Python version:", sys.version)
4+
print("Python path:", sys.path)
5+
6+
7+
8+
from pycompss.dds import DDS
9+
#from dataclay import Client
10+
import dataclay
11+
#from pycompss.api.task import task
12+
13+
#@task()
14+
def _filter(dds):
15+
print("##Filter numbers##")
16+
even_numbers = dds.filter( lambda x : x % 2 == 0 ).collect()
17+
return even_numbers
18+
19+
def main():
20+
print(dir(dataclay))
21+
#client = Client(host="127.0.0.1", username="testuser", password="s3cret", dataset="testdata")
22+
#client.start()
23+
24+
print("##Creating DDS object and making it persistent##")
25+
data = range(10)
26+
dds = DDS().load(data)
27+
#dds.make_persistent("dds")
28+
29+
even_numbers=_filter(dds)
30+
31+
print("##Print##")
32+
print(even_numbers)
33+
34+
if __name__ =='__main__':
35+
main()
36+

‎examples/dds/docker-compose.yml‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
services:
2+
3+
redis:
4+
image: redis:latest
5+
ports:
6+
- 6379:6379
7+
8+
metadata-service:
9+
image: "ghcr.io/bsc-dom/dataclay:dev"
10+
depends_on:
11+
- redis
12+
ports:
13+
- 16587:16587
14+
environment:
15+
- DATACLAY_KV_HOST=redis
16+
- DATACLAY_KV_PORT=6379
17+
- DATACLAY_ID
18+
- DATACLAY_PASSWORD=s3cret
19+
- DATACLAY_USERNAME=testuser
20+
- DATACLAY_DATASET=testdata
21+
- DATACLAY_METADATA_PORT=16587
22+
- DATACLAY_LOGLEVEL=DEBUG
23+
command: python -m dataclay.metadata
24+
25+
backend:
26+
image: "ghcr.io/bsc-dom/dataclay:dev"
27+
depends_on:
28+
- redis
29+
ports:
30+
- 6867:6867
31+
environment:
32+
- DATACLAY_KV_HOST=redis
33+
- DATACLAY_KV_PORT=6379
34+
- DATACLAY_BACKEND_ID
35+
- DATACLAY_BACKEND_NAME
36+
- DATACLAY_BACKEND_PORT=6867
37+
- DATACLAY_LOGLEVEL=DEBUG
38+
command: python -m dataclay.backend
39+
volumes:
40+
#- ./model:/workdir/model:ro
41+
- ./pycompss/:/workdir/pycompss/:ro

‎examples/dds/master/client.py‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
from pycompss.dds import DDS
2+
#from dataclay import Client
3+
import dataclay
4+
from pycompss.api.task import task
5+
6+
@task(returns=1)
7+
def _filter(dds):
8+
print("##Filter numbers##")
9+
even_numbers = dds.filter( lambda x : x % 2 == 0 ).collect()
10+
return even_numbers
11+
12+
def main():
13+
print(dir(dataclay))
14+
#client = Client(host="127.0.0.1", username="testuser", password="s3cret", dataset="testdata")
15+
#client.start()
16+
17+
print("##Creating DDS object and making it persistent##")
18+
data = range(10)
19+
dds = DDS().load(data)
20+
dds.make_persistent("dds")
21+
22+
even_numbers=_filter(dds)
23+
24+
print("##Print##")
25+
print(even_numbers)
26+
27+
if __name__ =='__main__':
28+
main()

‎examples/dds/master/docker-compose.yml‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
services:
2+
3+
redis:
4+
image: redis:latest
5+
ports:
6+
- 6379:6379
7+
8+
metadata-service:
9+
image: "ghcr.io/bsc-dom/dataclay:dev"
10+
depends_on:
11+
- redis
12+
ports:
13+
- 16587:16587
14+
environment:
15+
- DATACLAY_KV_HOST=redis
16+
- DATACLAY_KV_PORT=6379
17+
- DATACLAY_ID
18+
- DATACLAY_PASSWORD=s3cret
19+
- DATACLAY_USERNAME=testuser
20+
- DATACLAY_DATASET=testdata
21+
- DATACLAY_METADATA_PORT=16587
22+
- DATACLAY_LOGLEVEL=DEBUG
23+
command: python -m dataclay.metadata
24+
25+
backend:
26+
image: "ghcr.io/bsc-dom/dataclay:dev"
27+
depends_on:
28+
- redis
29+
ports:
30+
- 6867:6867
31+
environment:
32+
- DATACLAY_KV_HOST=redis
33+
- DATACLAY_KV_PORT=6379
34+
- DATACLAY_BACKEND_ID
35+
- DATACLAY_BACKEND_NAME
36+
- DATACLAY_BACKEND_PORT=6867
37+
- DATACLAY_LOGLEVEL=DEBUG
38+
command: python -m dataclay.backend
39+
volumes:
40+
- ./model:/workdir/model:ro

0 commit comments

Comments
 (0)