Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Problem: Config profile could not be found. I need help using semaphore ui and AWS profiles from the ~/.aws/credentials. #2024

Open
kdjomeda opened this issue May 18, 2024 · 7 comments
Open

Problem: Config profile could not be found. I need help using semaphore ui and AWS profiles from the ~/.aws/credentials. #2024

kdjomeda opened this issue May 18, 2024 · 7 comments
Assignees
@tboerger @fiftin
Labels
needs clarification needs investigation

Comments

@kdjomeda
Copy link

kdjomeda commented May 18, 2024 •
edited

Hello Community,

I have run into issues trying to execute ansible-semaphoreui with AWS profiles. There are 2 issues in one:

  1. Semaphore ui doesn't seem to see the AWS PROFILES from the installed awscli from python with ~/.aws/{credentials,config} configured for the same user that owns and executes semaphores
  2. If the above is fixed, how to pass AWS_PROFILE=blablabla command line environment variable to the ansible-playbook -I inventory blabla.playbook.yml bla bla bla execution ?

Setup used
ubuntu 22.04 LTS on AWS EC2

semaphore@ip-172-18-1-143:~$ whoami
semaphore
semaphore@ip-172-18-1-143:~$ ls
LICENSE  ansible_projects  config.json  semaphore  semaphore_2.9.64_linux_arm64.tar.gz  test
semaphore@ip-172-18-1-143:~$ python --version
Python 3.12.0
semaphore@ip-172-18-1-143:~$ ansible --version
ansible [core 2.16.6]
  config file = None
  configured module search path = ['/home/semaphore/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/semaphore/.pyenv/versions/3.12.0/lib/python3.12/site-packages/ansible
  ansible collection location = /home/semaphore/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/semaphore/.pyenv/versions/3.12.0/bin/ansible
  python version = 3.12.0 (main, Apr 26 2024, 22:47:02) [GCC 13.2.0] (/home/semaphore/.pyenv/versions/3.12.0/bin/python3.12)
  jinja version = 3.1.3
  libyaml = True
semaphore@ip-172-18-1-143:~$ pip freeze | grep -e awscli -e boto
awscli==1.32.93
boto==2.49.0
boto3==1.34.93
botocore==1.34.93
semaphore@ip-172-18-1-143:~$ ls -lh ~/.aws/
total 12K
drwxrwxr-x 3 semaphore semaphore 4.0K May  7 15:02 cli
-rw------- 1 semaphore semaphore  695 Apr 26 23:15 config
-rw------- 1 semaphore semaphore  116 Apr 26 23:15 credentials
semaphore@ip-172-18-1-143:~$ aws s3 ls s3://devops-data-sync-test/ --profile devops --region us-east-1
                           PRE AWS:ComplianceItem/
                           PRE AWS:ComplianceSummary/
                           PRE AWS:InstanceInformation/
                           PRE AWS:Tag/
semaphore@ip-172-18-1-143:~$ ./semaphore version
v2.9.64
semaphore@ip-172-18-1-143:~$ ps aux | grep semaphore
semapho+    2686  0.0  0.3 1258012 29864 ?       Ssl  May07   1:13 /home/semaphore/semaphore server --config=/home/semaphore/config.json
root       68544  0.0  0.0  17852  6272 pts/2    S+   08:40   0:00 sudo -i -u semaphore
root       68560  0.0  0.0  17852  2540 pts/0    Ss   08:40   0:00 sudo -i -u semaphore
semapho+   71159  0.0  0.0   6676  1920 pts/0    S+   09:02   0:00 grep --color=auto semaphore
Screenshot 2024-05-18 at 08 52 10 My task template is as follows Screenshot_2024-05-18_at_09_08_02

with the CLI args

["-e","global_var_environment=prod","-e", "global_var_profile=prod-uat-example","-vvvv"]

I had to pass the aws profile as a extra argument passed to ansible playbook shown below:

- name: All Fact Gatherer playbook
  hosts: localhost
  connection: local
  gather_facts: true
  vars_files:
    - "playbook_vars/{{ global_var_environment }}/global_vars.yml"
    - "playbook_vars/{{ global_var_environment }}/vault/{{ global_var_environment }}_variables.vault"
  environment:
    AWS_PROFILE: "{{ global_var_profile | default(lookup('env','AWS_PROFILE')) }}"
  tasks:

The default value is to pick it from the command line environment variable if I am able to set one

When I execute a task on it in debug mode the output is this

Task 12 added to queue
2:39:35 PM
Started: 12
2:39:35 PM
Run TaskRunner with template: Fact_Gatherer_Playbook_Test
2:39:35 PM
Preparing: 12
2:39:35 PM
Warning: Permanently added 'github.com' (ED25519) to the list of known hosts.
2:39:36 PM
From github.com:companyA/ansible_project_project_a
2:39:36 PM
   9ef3a5d..8d380a5  semaphore  -> origin/semaphore
2:39:36 PM
Updating Repository git@github.com:companyA/ansible_project_project_a.git
2:39:36 PM
Warning: Permanently added 'github.com' (ED25519) to the list of known hosts.
2:39:36 PM
From github.com:companyA/ansible_project_project_a
2:39:36 PM
 * branch            semaphore  -> FETCH_HEAD
2:39:36 PM
Updating 9ef3a5d..8d380a5
2:39:36 PM
Fast-forward
2:39:36 PM
 fact_gatherer_playbook.yml | 2 +-
2:39:36 PM
 1 file changed, 1 insertion(+), 1 deletion(-)
2:39:36 PM
No collections/requirements.yml file found. Skip galaxy install process.
2:39:36 PM
roles/requirements.yml has no changes. Skip galaxy install process.
2:39:36 PM
ansible-playbook [core 2.16.6]
2:39:36 PM
  config file = /home/semaphore/ansible_projects/repository_1_1/ansible.cfg
2:39:36 PM
  configured module search path = ['/home/semaphore/ansible_projects/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
2:39:36 PM
  ansible python module location = /home/semaphore/.pyenv/versions/3.12.0/lib/python3.12/site-packages/ansible
2:39:36 PM
  ansible collection location = /home/semaphore/ansible_projects/repository_1_1/collections
2:39:36 PM
  executable location = /home/semaphore/.pyenv/versions/3.12.0/bin/ansible-playbook
2:39:36 PM
  python version = 3.12.0 (main, Apr 26 2024, 22:47:02) [GCC 13.2.0] (/home/semaphore/.pyenv/versions/3.12.0/bin/python3.12)
2:39:36 PM
  jinja version = 3.1.3
2:39:36 PM
  libyaml = True
2:39:36 PM
Using /home/semaphore/ansible_projects/repository_1_1/ansible.cfg as config file
2:39:36 PM
[DEPRECATION WARNING]: [defaults]collections_paths option, does not fit var 
2:39:36 PM
naming standard, use the singular form collections_path instead. This feature 
2:39:36 PM
will be removed from ansible-core in version 2.19. Deprecation warnings can be 
2:39:36 PM
disabled by setting deprecation_warnings=False in ansible.cfg.
2:39:36 PM
setting up inventory plugins
2:39:37 PM
Loading collection ansible.builtin from 
2:39:37 PM
redirecting (type: inventory) ansible.builtin.aws_ec2 to amazon.aws.aws_ec2
2:39:37 PM
Loading collection amazon.aws from /home/semaphore/.pyenv/versions/3.12.0/lib/python3.12/site-packages/ansible_collections/amazon/aws
2:39:39 PM
Parsed /home/semaphore/ansible_projects/repository_1_1/inventory/bb.aws_ec2.yml inventory source with ansible_collections.amazon.aws.plugins.inventory.aws_ec2 plugin
2:39:39 PM
setting up inventory plugins
2:39:39 PM
redirecting (type: inventory) ansible.builtin.aws_ec2 to amazon.aws.aws_ec2
2:39:39 PM
ansible_collections.amazon.aws.plugins.inventory.aws_ec2 declined parsing /home/semaphore/ansible_projects/repository_1_1/inventory/bb.aws_rds.yml as it did not pass its verify_file() method
2:39:39 PM
host_list declined parsing /home/semaphore/ansible_projects/repository_1_1/inventory/bb.aws_rds.yml as it did not pass its verify_file() method
2:39:39 PM
script declined parsing /home/semaphore/ansible_projects/repository_1_1/inventory/bb.aws_rds.yml as it did not pass its verify_file() method
2:39:39 PM
Using inventory plugin 'ansible_collections.amazon.aws.plugins.inventory.aws_rds' to process inventory source '/home/semaphore/ansible_projects/repository_1_1/inventory/bb.aws_rds.yml'
2:39:39 PM
toml declined parsing /home/semaphore/ansible_projects/repository_1_1/inventory/bb.aws_rds.yml as it did not pass its verify_file() method
2:39:39 PM
[WARNING]:  * Failed to parse
2:39:39 PM
/home/semaphore/ansible_projects/repository_1_1/inventory/bb.aws_rds.yml with
2:39:39 PM
auto plugin: The config profile (prod-uat-example) could not be found
2:39:39 PM
  File "/home/semaphore/.pyenv/versions/3.12.0/lib/python3.12/site-packages/ansible/inventory/manager.py", line 293, in parse_source
2:39:39 PM
    plugin.parse(self._inventory, self._loader, source, cache=cache)
2:39:39 PM
  File "/home/semaphore/.pyenv/versions/3.12.0/lib/python3.12/site-packages/ansible/plugins/inventory/auto.py", line 59, in parse
2:39:39 PM
    plugin.parse(inventory, loader, path, cache=cache)
2:39:39 PM
  File "/home/semaphore/.pyenv/versions/3.12.0/lib/python3.12/site-packages/ansible_collections/amazon/aws/plugins/inventory/aws_rds.py", line 282, in parse
2:39:39 PM
    results = self._get_all_db_hosts(
2:39:39 PM
              ^^^^^^^^^^^^^^^^^^^^^^^
2:39:39 PM
  File "/home/semaphore/.pyenv/versions/3.12.0/lib/python3.12/site-packages/ansible_collections/amazon/aws/plugins/inventory/aws_rds.py", line 253, in _get_all_db_hosts
2:39:39 PM
    for connection, _region in self.all_clients("rds"):
2:39:39 PM
  File "/home/semaphore/.pyenv/versions/3.12.0/lib/python3.12/site-packages/ansible_collections/amazon/aws/plugins/plugin_utils/inventory.py", line 178, in all_clients
2:39:39 PM
    connection = self.client(service, region=region)
2:39:39 PM
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2:39:39 PM
  File "/home/semaphore/.pyenv/versions/3.12.0/lib/python3.12/site-packages/ansible_collections/amazon/aws/plugins/plugin_utils/inventory.py", line 84, in client
2:39:39 PM
    return super().client(*args, **kw_args)
2:39:39 PM
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2:39:39 PM
  File "/home/semaphore/.pyenv/versions/3.12.0/lib/python3.12/site-packages/ansible_collections/amazon/aws/plugins/plugin_utils/base.py", line 41, in client
2:39:39 PM
    conn = boto3_conn(self, conn_type="client", resource=service, **kw_args)
2:39:39 PM
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2:39:39 PM
  File "/home/semaphore/.pyenv/versions/3.12.0/lib/python3.12/site-packages/ansible_collections/amazon/aws/plugins/plugin_utils/botocore.py", line 39, in boto3_conn
2:39:39 PM
    plugin.fail_aws(to_native(e))
2:39:39 PM
  File "/home/semaphore/.pyenv/versions/3.12.0/lib/python3.12/site-packages/ansible_collections/amazon/aws/plugins/plugin_utils/base.py", line 34, in fail_aws
2:39:39 PM
    self._do_fail(to_native(message))
2:39:39 PM
  File "/home/semaphore/.pyenv/versions/3.12.0/lib/python3.12/site-packages/ansible_collections/amazon/aws/plugins/plugin_utils/base.py", line 28, in _do_fail
2:39:39 PM
    raise AnsibleError(message)
2:39:39 PM
  File "/home/semaphore/.pyenv/versions/3.12.0/lib/python3.12/site-packages/ansible/inventory/manager.py", line 293, in parse_source
2:39:39 PM
    plugin.parse(self._inventory, self._loader, source, cache=cache)
2:39:39 PM
  File "/home/semaphore/.pyenv/versions/3.12.0/lib/python3.12/site-packages/ansible/plugins/inventory/yaml.py", line 114, in parse
2:39:39 PM
    raise AnsibleParserError('Plugin configuration YAML file, not YAML inventory')
2:39:39 PM
  File "/home/semaphore/.pyenv/versions/3.12.0/lib/python3.12/site-packages/ansible/inventory/manager.py", line 293, in parse_source
2:39:39 PM
    plugin.parse(self._inventory, self._loader, source, cache=cache)
2:39:39 PM
  File "/home/semaphore/.pyenv/versions/3.12.0/lib/python3.12/site-packages/ansible/plugins/inventory/ini.py", line 138, in parse
2:39:39 PM
    raise AnsibleParserError(e)
2:39:39 PM
[WARNING]:  * Failed to parse
2:39:39 PM
/home/semaphore/ansible_projects/repository_1_1/inventory/bb.aws_rds.yml with
2:39:39 PM
yaml plugin: Plugin configuration YAML file, not YAML inventory
2:39:39 PM
[WARNING]:  * Failed to parse
2:39:39 PM
/home/semaphore/ansible_projects/repository_1_1/inventory/bb.aws_rds.yml with
2:39:39 PM
ini plugin: Invalid host pattern 'plugin:' supplied, ending in ':' is not
2:39:39 PM
allowed, this character is reserved to provide a port.
2:39:39 PM
[WARNING]: Unable to parse
2:39:39 PM
/home/semaphore/ansible_projects/repository_1_1/inventory/bb.aws_rds.yml as an
2:39:39 PM
inventory source
2:39:39 PM
[WARNING]: provided hosts list is empty, only localhost is available. Note that
2:39:39 PM
the implicit localhost does not match 'all'
2:39:39 PM
redirecting (type: modules) ansible.builtin.aws_caller_info to amazon.aws.aws_caller_info
2:39:39 PM
redirecting (type: modules) ansible.builtin.ec2_instance_info to community.aws.ec2_instance_info
2:39:39 PM
Loading collection community.aws from /home/semaphore/.pyenv/versions/3.12.0/lib/python3.12/site-packages/ansible_collections/community/aws
2:39:39 PM
redirecting (type: modules) community.aws.ec2_instance_info to amazon.aws.ec2_instance_info
2:39:39 PM
redirecting (type: modules) ansible.builtin.ec2_instance_info to community.aws.ec2_instance_info
2:39:39 PM
redirecting (type: modules) community.aws.ec2_instance_info to amazon.aws.ec2_instance_info
2:39:39 PM
redirecting (type: modules) ansible.builtin.ec2_instance_info to community.aws.ec2_instance_info
2:39:39 PM
redirecting (type: modules) community.aws.ec2_instance_info to amazon.aws.ec2_instance_info
2:39:39 PM
redirecting (type: modules) ansible.builtin.ec2_instance_info to community.aws.ec2_instance_info
2:39:39 PM
redirecting (type: modules) community.aws.ec2_instance_info to amazon.aws.ec2_instance_info
2:39:39 PM
redirecting (type: modules) ansible.builtin.ec2_instance_info to community.aws.ec2_instance_info
2:39:39 PM
redirecting (type: modules) community.aws.ec2_instance_info to amazon.aws.ec2_instance_info
2:39:39 PM
redirecting (type: modules) ansible.builtin.ec2_instance_info to community.aws.ec2_instance_info
2:39:39 PM
redirecting (type: modules) community.aws.ec2_instance_info to amazon.aws.ec2_instance_info
2:39:39 PM
redirecting (type: modules) ansible.builtin.ec2_vpc_net_info to amazon.aws.ec2_vpc_net_info
2:39:39 PM
redirecting (type: modules) ansible.builtin.ec2_vpc_subnet_info to amazon.aws.ec2_vpc_subnet_info
2:39:39 PM
redirecting (type: modules) ansible.builtin.ec2_vpc_subnet_info to amazon.aws.ec2_vpc_subnet_info
2:39:39 PM
redirecting (type: modules) ansible.builtin.ec2_vpc_subnet_info to amazon.aws.ec2_vpc_subnet_info
2:39:39 PM
redirecting (type: modules) ansible.builtin.ec2_vpc_subnet_info to amazon.aws.ec2_vpc_subnet_info
2:39:39 PM
redirecting (type: modules) ansible.builtin.elb_application_lb_info to community.aws.elb_application_lb_info
2:39:39 PM
redirecting (type: modules) community.aws.elb_application_lb_info to amazon.aws.elb_application_lb_info
2:39:39 PM
redirecting (type: modules) ansible.builtin.iam_role_info to community.aws.iam_role_info
2:39:39 PM
redirecting (type: modules) community.aws.iam_role_info to amazon.aws.iam_role_info
2:39:39 PM
redirecting (type: modules) ansible.builtin.elb_application_lb_info to community.aws.elb_application_lb_info
2:39:39 PM
redirecting (type: modules) community.aws.elb_application_lb_info to amazon.aws.elb_application_lb_info
2:39:39 PM
redirecting (type: modules) ansible.builtin.elb_application_lb_info to community.aws.elb_application_lb_info
2:39:39 PM
redirecting (type: modules) community.aws.elb_application_lb_info to amazon.aws.elb_application_lb_info
2:39:39 PM
redirecting (type: modules) ansible.builtin.elb_application_lb_info to community.aws.elb_application_lb_info
2:39:39 PM
redirecting (type: modules) community.aws.elb_application_lb_info to amazon.aws.elb_application_lb_info
2:39:39 PM
redirecting (type: modules) ansible.builtin.cloudwatchlogs_log_group_info to community.aws.cloudwatchlogs_log_group_info
2:39:39 PM
redirecting (type: modules) community.aws.cloudwatchlogs_log_group_info to amazon.aws.cloudwatchlogs_log_group_info
2:39:39 PM
redirecting (type: modules) ansible.builtin.ec2_vpc_route_table_info to community.aws.ec2_vpc_route_table_info
2:39:39 PM
redirecting (type: modules) community.aws.ec2_vpc_route_table_info to amazon.aws.ec2_vpc_route_table_info
2:39:39 PM
redirecting (type: modules) ansible.builtin.ec2_vpc_route_table_info to community.aws.ec2_vpc_route_table_info
2:39:39 PM
redirecting (type: modules) community.aws.ec2_vpc_route_table_info to amazon.aws.ec2_vpc_route_table_info
2:39:39 PM
Loading callback plugin default of type stdout, v2.0 from /home/semaphore/.pyenv/versions/3.12.0/lib/python3.12/site-packages/ansible/plugins/callback/default.py
2:39:39 PM
Skipping callback 'default', as we already have a stdout callback.
2:39:39 PM
Skipping callback 'minimal', as we already have a stdout callback.
2:39:39 PM
Skipping callback 'oneline', as we already have a stdout callback.
2:39:39 PM
2:39:39 PM
PLAYBOOK: fact_gatherer_playbook.yml *******************************************
2:39:39 PM
Positional arguments: fact_gatherer_playbook.yml
2:39:39 PM
verbosity: 4
2:39:39 PM
connection: ssh
2:39:39 PM
become_method: sudo
2:39:39 PM
tags: ('all',)
2:39:39 PM
inventory: ('/home/semaphore/ansible_projects/repository_1_1/inventory',)
2:39:39 PM
extra_vars: ('{"semaphore_vars":{"task_details":{"id":12,"username":"joseph@companyA.app"}}}', 'global_var_environment=prod', 'global_var_profile=prod-uat-example')
2:39:39 PM
vault_password_files: ('/home/semaphore/ansible_projects/access_key_1152535',)
2:39:39 PM
forks: 5
2:39:39 PM
1 plays in fact_gatherer_playbook.yml
2:39:39 PM
Read vars_file 'playbook_vars/{{ global_var_environment }}/global_vars.yml'
2:39:39 PM
Trying secret FileVaultSecret(filename='/home/semaphore/ansible_projects/access_key_1152535') for vault_id=default
2:39:39 PM
Read vars_file 'playbook_vars/{{ global_var_environment }}/vault/{{ global_var_environment }}_variables.vault'
2:39:39 PM
Read vars_file 'playbook_vars/{{ global_var_environment }}/global_vars.yml'
2:39:39 PM
Trying secret FileVaultSecret(filename='/home/semaphore/ansible_projects/access_key_1152535') for vault_id=default
2:39:39 PM
Read vars_file 'playbook_vars/{{ global_var_environment }}/vault/{{ global_var_environment }}_variables.vault'
2:39:39 PM
Read vars_file 'playbook_vars/{{ global_var_environment }}/global_vars.yml'
2:39:39 PM
Trying secret FileVaultSecret(filename='/home/semaphore/ansible_projects/access_key_1152535') for vault_id=default
2:39:39 PM
Read vars_file 'playbook_vars/{{ global_var_environment }}/vault/{{ global_var_environment }}_variables.vault'
2:39:39 PM
2:39:39 PM
PLAY [All Fact Gatherer playbook] **********************************************
2:39:39 PM
Read vars_file 'playbook_vars/{{ global_var_environment }}/global_vars.yml'
2:39:39 PM
Trying secret FileVaultSecret(filename='/home/semaphore/ansible_projects/access_key_1152535') for vault_id=default
2:39:39 PM
Read vars_file 'playbook_vars/{{ global_var_environment }}/vault/{{ global_var_environment }}_variables.vault'
2:39:39 PM
2:39:39 PM
TASK [Gathering Facts] *********************************************************
2:39:39 PM
task path: /home/semaphore/ansible_projects/repository_1_1/fact_gatherer_playbook.yml:1
2:39:39 PM
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: semaphore
2:39:39 PM
<127.0.0.1> EXEC /bin/sh -c 'echo ~semaphore && sleep 0'
2:39:39 PM
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/semaphore/.ansible/tmp `"&& mkdir "` echo /home/semaphore/.ansible/tmp/ansible-tmp-1715092779.3082068-10327-3425972425968 `" && echo ansible-tmp-1715092779.3082068-10327-3425972425968="` echo /home/semaphore/.ansible/tmp/ansible-tmp-1715092779.3082068-10327-3425972425968 `" ) && sleep 0'
2:39:40 PM
Using module file /home/semaphore/.pyenv/versions/3.12.0/lib/python3.12/site-packages/ansible/modules/setup.py
2:39:40 PM
<127.0.0.1> PUT /home/semaphore/ansible_projects/.ansible/tmp/ansible-local-10275oprb0_on/tmp7271e671 TO /home/semaphore/.ansible/tmp/ansible-tmp-1715092779.3082068-10327-3425972425968/AnsiballZ_setup.py
2:39:40 PM
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/semaphore/.ansible/tmp/ansible-tmp-1715092779.3082068-10327-3425972425968/ /home/semaphore/.ansible/tmp/ansible-tmp-1715092779.3082068-10327-3425972425968/AnsiballZ_setup.py && sleep 0'
2:39:40 PM
<127.0.0.1> EXEC /bin/sh -c 'AWS_PROFILE=prod-uat-example /home/semaphore/.pyenv/versions/3.12.0/bin/python3.12 /home/semaphore/.ansible/tmp/ansible-tmp-1715092779.3082068-10327-3425972425968/AnsiballZ_setup.py && sleep 0'
2:39:41 PM
<127.0.0.1> EXEC /bin/sh -c 'rm -f -r /home/semaphore/.ansible/tmp/ansible-tmp-1715092779.3082068-10327-3425972425968/ > /dev/null 2>&1 && sleep 0'
2:39:41 PM
ok: [localhost]
2:39:41 PM
Read vars_file 'playbook_vars/{{ global_var_environment }}/global_vars.yml'
2:39:41 PM
Trying secret FileVaultSecret(filename='/home/semaphore/ansible_projects/access_key_1152535') for vault_id=default
2:39:41 PM
Read vars_file 'playbook_vars/{{ global_var_environment }}/vault/{{ global_var_environment }}_variables.vault'
2:39:41 PM
Read vars_file 'playbook_vars/{{ global_var_environment }}/global_vars.yml'
2:39:41 PM
Trying secret FileVaultSecret(filename='/home/semaphore/ansible_projects/access_key_1152535') for vault_id=default
2:39:41 PM
Read vars_file 'playbook_vars/{{ global_var_environment }}/vault/{{ global_var_environment }}_variables.vault'
2:39:41 PM
Read vars_file 'playbook_vars/{{ global_var_environment }}/global_vars.yml'
2:39:41 PM
Trying secret FileVaultSecret(filename='/home/semaphore/ansible_projects/access_key_1152535') for vault_id=default
2:39:41 PM
Read vars_file 'playbook_vars/{{ global_var_environment }}/vault/{{ global_var_environment }}_variables.vault'
2:39:41 PM
2:39:41 PM
TASK [Obtaining AWS Account Facts] *********************************************
2:39:41 PM
task path: /home/semaphore/ansible_projects/repository_1_1/fact_gatherer_playbook.yml:11
2:39:41 PM
redirecting (type: modules) ansible.builtin.aws_caller_info to amazon.aws.aws_caller_info
2:39:41 PM
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: semaphore
2:39:41 PM
<127.0.0.1> EXEC /bin/sh -c 'echo ~semaphore && sleep 0'
2:39:41 PM
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/semaphore/.ansible/tmp `"&& mkdir "` echo /home/semaphore/.ansible/tmp/ansible-tmp-1715092780.7218232-10425-79984230110935 `" && echo ansible-tmp-1715092780.7218232-10425-79984230110935="` echo /home/semaphore/.ansible/tmp/ansible-tmp-1715092780.7218232-10425-79984230110935 `" ) && sleep 0'
2:39:41 PM
redirecting (type: modules) ansible.builtin.aws_caller_info to amazon.aws.aws_caller_info
2:39:41 PM
Using module file /home/semaphore/.pyenv/versions/3.12.0/lib/python3.12/site-packages/ansible_collections/amazon/aws/plugins/modules/aws_caller_info.py
2:39:41 PM
<127.0.0.1> PUT /home/semaphore/ansible_projects/.ansible/tmp/ansible-local-10275oprb0_on/tmp5zi4g4ij TO /home/semaphore/.ansible/tmp/ansible-tmp-1715092780.7218232-10425-79984230110935/AnsiballZ_aws_caller_info.py
2:39:41 PM
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/semaphore/.ansible/tmp/ansible-tmp-1715092780.7218232-10425-79984230110935/ /home/semaphore/.ansible/tmp/ansible-tmp-1715092780.7218232-10425-79984230110935/AnsiballZ_aws_caller_info.py && sleep 0'
2:39:41 PM
<127.0.0.1> EXEC /bin/sh -c 'AWS_PROFILE=prod-uat-example /home/semaphore/.pyenv/versions/3.12.0/bin/python3.12 /home/semaphore/.ansible/tmp/ansible-tmp-1715092780.7218232-10425-79984230110935/AnsiballZ_aws_caller_info.py && sleep 0'
2:39:41 PM
<127.0.0.1> EXEC /bin/sh -c 'rm -f -r /home/semaphore/.ansible/tmp/ansible-tmp-1715092780.7218232-10425-79984230110935/ > /dev/null 2>&1 && sleep 0'
2:39:41 PM
The full traceback is:
2:39:41 PM
  File "/tmp/ansible_aws_caller_info_payload_9m7tag3n/ansible_aws_caller_info_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/botocore.py", line 92, in boto3_conn
2:39:41 PM
    return _boto3_conn(conn_type=conn_type, resource=resource, region=region, endpoint=endpoint, **params)
2:39:41 PM
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2:39:41 PM
  File "/tmp/ansible_aws_caller_info_payload_9m7tag3n/ansible_aws_caller_info_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/botocore.py", line 148, in _boto3_conn
2:39:41 PM
    session = boto3.session.Session(
2:39:41 PM
              ^^^^^^^^^^^^^^^^^^^^^^
2:39:41 PM
  File "/home/semaphore/.pyenv/versions/3.12.0/lib/python3.12/site-packages/boto3/session.py", line 90, in __init__
2:39:41 PM
    self._setup_loader()
2:39:41 PM
  File "/home/semaphore/.pyenv/versions/3.12.0/lib/python3.12/site-packages/boto3/session.py", line 131, in _setup_loader
2:39:41 PM
    self._loader = self._session.get_component('data_loader')
2:39:41 PM
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2:39:41 PM
  File "/home/semaphore/.pyenv/versions/3.12.0/lib/python3.12/site-packages/botocore/session.py", line 802, in get_component
2:39:41 PM
    return self._components.get_component(name)
2:39:41 PM
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2:39:41 PM
  File "/home/semaphore/.pyenv/versions/3.12.0/lib/python3.12/site-packages/botocore/session.py", line 1140, in get_component
2:39:41 PM
    self._components[name] = factory()
2:39:41 PM
                             ^^^^^^^^^
2:39:41 PM
  File "/home/semaphore/.pyenv/versions/3.12.0/lib/python3.12/site-packages/botocore/session.py", line 199, in 
2:39:41 PM
    lambda: create_loader(self.get_config_variable('data_path')),
2:39:41 PM
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2:39:41 PM
  File "/home/semaphore/.pyenv/versions/3.12.0/lib/python3.12/site-packages/botocore/session.py", line 323, in get_config_variable
2:39:41 PM
    return self.get_component('config_store').get_config_variable(
2:39:41 PM
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2:39:41 PM
  File "/home/semaphore/.pyenv/versions/3.12.0/lib/python3.12/site-packages/botocore/configprovider.py", line 465, in get_config_variable
2:39:41 PM
    return provider.provide()
2:39:41 PM
           ^^^^^^^^^^^^^^^^^^
2:39:41 PM
  File "/home/semaphore/.pyenv/versions/3.12.0/lib/python3.12/site-packages/botocore/configprovider.py", line 671, in provide
2:39:41 PM
    value = provider.provide()
2:39:41 PM
            ^^^^^^^^^^^^^^^^^^
2:39:41 PM
  File "/home/semaphore/.pyenv/versions/3.12.0/lib/python3.12/site-packages/botocore/configprovider.py", line 761, in provide
2:39:41 PM
    scoped_config = self._session.get_scoped_config()
2:39:41 PM
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2:39:41 PM
  File "/home/semaphore/.pyenv/versions/3.12.0/lib/python3.12/site-packages/botocore/session.py", line 422, in get_scoped_config
2:39:41 PM
    raise ProfileNotFound(profile=profile_name)
2:39:41 PM
fatal: [localhost]: FAILED! => {
2:39:41 PM
    "changed": false,
2:39:41 PM
    "invocation": {
2:39:41 PM
        "module_args": {
2:39:41 PM
            "access_key": null,
2:39:41 PM
            "aws_ca_bundle": null,
2:39:41 PM
            "aws_config": null,
2:39:41 PM
            "debug_botocore_endpoint_logs": false,
2:39:41 PM
            "endpoint_url": null,
2:39:41 PM
            "profile": "prod-uat-example",
2:39:41 PM
            "region": null,
2:39:41 PM
            "secret_key": null,
2:39:41 PM
            "session_token": null,
2:39:41 PM
            "validate_certs": true
2:39:41 PM
        }
2:39:41 PM
    },
2:39:41 PM
    "msg": "The config profile (prod-uat-example) could not be found"
2:39:41 PM
}
2:39:41 PM
2:39:41 PM
PLAY RECAP *********************************************************************
2:39:41 PM
localhost                  : ok=1    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0   
2:39:41 PM
2:39:42 PM
Running playbook failed: exit status 2

Grateful if anyone can share their wisdom on this. Been blocked at this since 24th April 2024. I think I have run out of options and wanted to check with the community if there is something I needed to change or some understanding I don't have yet. Thanks in advance
@kdjomeda kdjomeda changed the title I need help using semaphore ui and AWS profiles from the ~/.aws/credentials. [Config profile could not be found] .I need help using semaphore ui and AWS profiles from the ~/.aws/credentials. May 18, 2024
@kdjomeda kdjomeda changed the title [Config profile could not be found] .I need help using semaphore ui and AWS profiles from the ~/.aws/credentials. [Config profile could not be found]. I need help using semaphore ui and AWS profiles from the ~/.aws/credentials. May 18, 2024
@kdjomeda kdjomeda changed the title [Config profile could not be found]. I need help using semaphore ui and AWS profiles from the ~/.aws/credentials. Problem: Config profile could not be found. I need help using semaphore ui and AWS profiles from the ~/.aws/credentials. May 18, 2024
@fiftin fiftin assigned fiftin and tboerger and unassigned fiftin May 21, 2024
@fiftin
Copy link
Collaborator

fiftin commented May 21, 2024 •
edited

Hi @kdjomeda does it work by using ansible-playbook? It is not looks like Semaphore issue because python was found correctly.

@fiftin fiftin self-assigned this May 21, 2024
@fiftin
Copy link
Collaborator

fiftin commented May 21, 2024 •
edited

@kdjomeda in your setup description you use AWS profile devops but in Ansible you use profile prod-uat-example. Is it correct? Can you try global_var_profile=devops ?

@kdjomeda
Copy link
Author

kdjomeda commented May 21, 2024 •
edited

Sorry that was a way of showing that profile was set up and operational. There are multiple profiles in there
Screenshot 2024-05-21 at 13 34 43

I am going to give it a try. I edited the profile to be prod-uat-example because it will reveal the product name. it is originally product-. I also used devops in the example so that it doesn't show critical names. But I have made a copy and we now have a profile called product-example. I will keep all consistent, try again and revert

@kdjomeda
Copy link
Author

@fiftin I did run it directly on the device and it run but not on the semaphoreui
Screenshot 2024-05-21 at 12 32 40

Screenshot 2024-05-21 at 12 34 25
Screenshot 2024-05-21 at 12 35 00

@fiftin
Copy link
Collaborator

fiftin commented May 21, 2024

Hm, it is really strange. @tboerger, @andreas-marschke do you have any idea?

@tboerger
Copy link
Collaborator

I don't have an idea really... I never used profiles.

@fiftin
Copy link
Collaborator

fiftin commented May 24, 2024

@kdjomeda Looks like I understood why it can't find your profile. Because Semaphore override HOME environment variable.
Please try following config which I got from ChatGPT:

- name: Launch EC2 instance
  hosts: localhost
  gather_facts: no
  environment:
    AWS_PROFILE: your-custom-profile-name
    AWS_CONFIG_FILE: /path/to/your/custom/config
    AWS_SHARED_CREDENTIALS_FILE: /path/to/your/custom/credentials
  tasks:
    - name: Create an EC2 instance
      ec2:
        key_name: my-key
        instance_type: t2.micro
        image: ami-12345678
        region: us-east-1
        wait: yes
      register: ec2
    - debug:
        var: ec2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tboerger tboerger

@fiftin fiftin

Labels
needs clarification needs investigation
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@tboerger @kdjomeda @fiftin