본문 바로가기
카테고리 없음

python 에서 .exe 배포- 3 ( s3 )

그냥 케이 2022. 7. 25. 20:35

 

안녕하세요 !

pyupdater 배포 관련 하여 세번째 글남깁니다

 

이전까지는 SCP 를 사용하여 특정 웹서버에 upload 하는 방법을 사용해봤엇는데 이번에 cloud를 aws로 옮기면서 S3 를 사용하기로하여 진행해보게되었습니다.

 

기본 셋팅은 이전과 같습니다. 아래 방법까지 됫다는 가정하에 scp 를 s3로 변경해볼게요

 

https://programmerk.tistory.com/67

 

python 에서 .exe 배포- 2 ( pyinstaller ,pyupdater,scp )

이글에 이어서 작성합니다. 아래글은 약 6개월전을 회상하면서 작성한글입니다. 이글은 현재 시점의 글입니다. https://programmerk.tistory.com/66 잘쓰던 프로그램이 결국 문제가생겻다. 서버 버전과 현

programmerk.tistory.com

 

 

 

할 목록은 다음과 같습니다

 

1. 기존 scp 설정 삭제

2. 기존 처럼 pyupdater setting 

3. pyupdater 의 s3 plugin 설치

4. 환경변수 설정 ( PYU_AWS_ID , PYU_AWS_SECRET )

5. pyupdater settings --plugin s3 설정

6. 배포

 

 

시작해볼까요

 

1. 기존 프로젝트의 폴더와 파일중

.pyupdater

pyu-data

client_config.py

 

이 세가지를 삭제해주세요

 

2. pyupdater setting 

>> pyupdater keys -c
 
PyUpdater %s 4.0
args %s None
args %s ['keys', '-c']
538 DEBUG: PyUpdater config data folder is missing
[ERROR] Not a PyUpdater repo: You must initialize your repository first
539 ERROR: Not a PyUpdater repo: You must initialize your repository first
Please enter app name - No Default Available
--> app
You entered app, is this correct?
[N/y]?y
3154 DEBUG: off_pri type: <class 'str'>
[INFO] Keypack placed in cwd
3157 INFO: Keypack placed in cwd

 

>> pyupdater init

PyUpdater %s 4.0
args %s None
args %s ['init']
Please enter app name
[DEFAULT] -> PyUpdater App
Press Enter To Use Default
--> app
You entered app, is this correct?
[N/y]?y
Please enter your name or company name
[DEFAULT] -> PyUpdater App
Press Enter To Use Default
--> 
You entered , is this correct?
[N/y]?y
Enter a url to ping for updates. - No Default Available
--> https://[your_bucket_name].s3.[your_region].amazonaws.com/[your_bucket_folder]/           
You entered https://[your_bucket_name].s3.[your_region].amazonaws.com/[your_bucket_folder]/, is this correct?
[N/y]?y
Would you like to add another url for backup?
[N/y]?n
Would you like to enable patch updates?
[Y/n]?
Please enter the path to where pyupdater will write the client_config.py file. You'll need to import this file to initialize the update process. 
Examples:

lib/utils, src/lib, src.

Leave blank to use the current directory
[DEFAULT] -> updateTest
Press Enter To Use Default
-->
You entered updateTest, is this correct?
[N/y]?y
[INFO] Creating pyu-data dir...
.
.
.
성공

 

update url 입력하는 부분에는 s3 정보를 입력하면 됩니다

your_bucket_name = s3 버킷 이름

your_region =  s3 버킷 지역

your_bucket_folder = s3 버킷에서 생성한 폴더 이름입니다.

 

 

 

>>  pyupdater keys -i

PyUpdater %s 4.0
args %s None
args %s ['keys', '-i']
520 DEBUG: Config Dir: C:\Users\ansgy\\main\python\updateTest\.pyupdater
521 DEBUG: Config DB: C:\Users\ansgy\\main\python\updateTest\.pyupdater\config.pyu
522 DEBUG: Config Dir: C:\Users\ansgy\\main\python\updateTest\.pyupdater
553 DEBUG: Config DB: C:\Users\ansgy\\main\python\updateTest\.pyupdater\config.pyu
554 DEBUG: Syncing db to filesystem
[INFO] Keypack import successfully
555 INFO: Keypack import successfully
557 DEBUG: Saving Config
557 DEBUG: Syncing db to filesystem
559 DEBUG: Config saved
560 DEBUG: Writing client_config.py
560 DEBUG: Adding PUBLIC_KEY to client_config.py
560 DEBUG: Adding APP_NAME to client_config.py
562 DEBUG: Adding COMPANY_NAME to client_config.py
564 DEBUG: Adding HTTP_TIMEOUT to cilent_config.py
565 DEBUG: Adding MAX_DOWNLOAD_RETRIES to client_config.py
565 DEBUG: Adding UPDATE_URLS to client_config.py
566 DEBUG: Wrote client config
PS C:\Users\ansgy\\main\python\updateTest>

 

 

 

>>  pyupdater make-spec app.py


PyUpdater %s 4.0
args %s None
args %s ['make-spec', 'app.py']
518 DEBUG: Config Dir: C:\Users\ansgy\\main\python\updateTest\.pyupdater
519 DEBUG: Config DB: C:\Users\ansgy\\main\python\updateTest\.pyupdater\config.pyu
521 DEBUG: Building from python source file: app.py
521 DEBUG: Source file abs path: C:\Users\ansgy\\main\python\updateTest\app.py
522 DEBUG: App Info: {'type': 'script', 'name': 'C:\\Users\\ansgy\\\\main\\python\\updateTest\\app.py'}
522 DEBUG: *** User generated spec file ***
549 DEBUG: There could be errors
549 DEBUG: Hook directory: c:\users\ansgy\appdata\local\programs\python\python38\lib\site-packages\pyupdater\hooks
550 DEBUG: Make spec cmd: --name=win --specpath=C:\Users\ansgy\\main\python\updateTest --additional-hooks-dir=c:\users\ansgy\appdata\local\programs\python\python38\lib\site-packages\pyupdater\hooks --log-level=ERROR C:\Users\ansgy\\main\python\updateTest\app.py
556 DEBUG: wrote C:\Users\ansgy\\main\python\updateTest\win.spec
[INFO] Spec file placed in cwd
557 INFO: Spec file placed in cwd

 

 

>>> pyupdater build --app-version=1.0.0 win.spec



PyUpdater %s 4.0
args %s None
args %s ['build', '--app-version=1.0.0', 'win.spec']
[INFO] Compiling...
594 INFO: Compiling...
597 DEBUG: Config Dir: C:\Users\ansgy\\main\python\updateTest\.pyupdater
.
.
.
성공

 

 

 

>>>> pyupdater pkg -p -s


PyUpdater %s 4.0
args %s None
args %s ['pkg', '-p', '-s']
568 DEBUG: Config Dir: C:\Users\ansgy\\main\python\updateTest\.pyupdater
569 DEBUG: Config DB: C:\Users\ansgy\\main\python\updateTest\.pyupdater\config.pyu
570 DEBUG: Config Dir: C:\Users\ansgy\\main\python\updateTest\.pyupdater
570 DEBUG: Config DB: C:\Users\ansgy\\main\python\updateTest\.pyupdater\config.pyu
571 DEBUG: Config Dir: C:\Users\ansgy\\main\python\updateTest\.pyupdater
571 DEBUG: Config DB: C:\Users\ansgy\\main\python\updateTest\.pyupdater\config.pyu
573 DEBUG: Config Dir: C:\Users\ansgy\\main\python\updateTest\.pyupdater
574 DEBUG: Config DB: C:\Users\ansgy\\main\python\updateTest\.pyupdater\config.pyu
[WARNING] Version file not found
574 WARNING: Version file not found
576 DEBUG: Created new version file
576 DEBUG: Creating new config file

.
.
.
성공

 

 

여기까지가 기본적인 설정 및 빌드 , 키 사이닝 입니다.

 

여기서부터 조금달라요 

 

 

3. s3 plugin 설치

$ pip install PyUpdater-S3-plugin

 

 

4.환경변수 설정 ( PYU_AWS_ID , PYU_AWS_SECRET )

PYU_AWS_ID 와 PYU_AWS_SECRET 키는 aws의 IAM 에서 만들어주면됩니다.

IAM 만들때 JSON 아래처럼 붙여넣으셔도되요

 

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": "s3:*",
            "Resource": "arn:aws:s3:::[your_bucket_name]/[your_bucket_folder]/*"
        }
    ]
}

 

 

 

 5. pyupdater settings --plugin s3

>> pyupdater settings --plugin s3


PyUpdater %s 4.0
args %s None
args %s ['settings', '--plugin', 's3']
557 DEBUG: Config Dir: C:\Users\ansgy\\main\python\updateTest\.pyupdater
558 DEBUG: Config DB: C:\Users\ansgy\\main\python\updateTest\.pyupdater\config.pyu
Please enter a bucket name - No Default Available
--> [bucket_name]
You entered [bucket_name], is this correct?
[N/y]?y
Please enter a bucket key - No Default Available
--> [bucket_folder]
You entered [bucket_folder], is this correct?
[N/y]?y
Please enter a bucket region - No Default Available
--> [bucket region]
You entered [bucket region], is this correct?
[N/y]?y
24867 DEBUG: Saving Config
24868 DEBUG: Syncing db to filesystem
24870 DEBUG: Config saved
24870 DEBUG: Writing client_config.py
24871 DEBUG: Adding PUBLIC_KEY to client_config.py
24871 DEBUG: Adding APP_NAME to client_config.py
24871 DEBUG: Adding COMPANY_NAME to client_config.py
24873 DEBUG: Adding HTTP_TIMEOUT to cilent_config.py
24873 DEBUG: Adding MAX_DOWNLOAD_RETRIES to client_config.py
24873 DEBUG: Adding UPDATE_URLS to client_config.py
24874 DEBUG: Wrote client config
[INFO] Saved config
24874 INFO: Saved config

 

 

6. 배포

 

배포를 딱하면 아래처럼 실패합니다 ㅎㅎ

>>> pyupdater upload --service s3


PyUpdater %s 4.0
args %s None
args %s ['upload', '--service', 's3']
529 DEBUG: Config Dir: C:\Users\ansgy\\main\python\updateTest\.pyupdater
530 DEBUG: Config DB: C:\Users\ansgy\\main\python\updateTest\.pyupdater\config.pyu
531 DEBUG: Config Dir: C:\Users\ansgy\\main\python\updateTest\.pyupdater
532 DEBUG: Config DB: C:\Users\ansgy\\main\python\updateTest\.pyupdater\config.pyu
533 DEBUG: Config Dir: C:\Users\ansgy\\main\python\updateTest\.pyupdater
533 DEBUG: Config DB: C:\Users\ansgy\\main\python\updateTest\.pyupdater\config.pyu
535 DEBUG: Config Dir: C:\Users\ansgy\\main\python\updateTest\.pyupdater
536 DEBUG: Config DB: C:\Users\ansgy\\main\python\updateTest\.pyupdater\config.pyu
>>>> {'bucket_name': '[bucket_name]', 'bucket_key': '[bucket_forder]', 'bucket_region': 'ap-northeast-2'}
region_name>>> ap-northeast-2
service_name>>> s3
965 DEBUG: Requested uploader: s3
[INFO] Starting upload
973 INFO: Starting upload
[INFO] Plugin: S3
974 INFO: Plugin: S3
[INFO] Author: Digital Sapphire
975 INFO: Author: Digital Sapphire


Uploading: app-win-1.0.0.zip - File 1 of 3

1085 ERROR: Failed to upload file
1213 DEBUG: app-win-1.0.0.zip failed to upload.  will retry


Uploading: keys.gz - File 1 of 3

1287 ERROR: Failed to upload file
1390 DEBUG: keys.gz failed to upload.  will retry


Uploading: versions.gz - File 1 of 3

1475 ERROR: Failed to upload file
1582 DEBUG: versions.gz failed to upload.  will retry


[ERROR] The following files were not uploaded
2199 ERROR: The following files were not uploaded
[ERROR] app-win-1.0.0.zip failed to upload
2203 ERROR: app-win-1.0.0.zip failed to upload
[ERROR] keys.gz failed to upload
2203 ERROR: keys.gz failed to upload
[ERROR] versions.gz failed to upload
2204 ERROR: versions.gz failed to upload
[ERROR] Upload failed!
2204 ERROR: Upload failed!

 

 

에러나는 부분이 

C:\Users\ansgy\AppData\Local\Programs\Python\Python38\Lib\site-packages\s3_uploader.py 의 upload_file 인데요

 

해당부분을 아래처럼 바꿔서

 

 

           self.s3.upload_file(

                filename,

                self.bucket_name,

                key,

                ExtraArgs={'ACL': 'public-read'},

                Callback=ProgressPercentage(filename)

            )

            log.debug('Uploaded {}'.format(filename))

            return True

        except Exception as err:

            log.error('Failed to upload file')

            log.error(err)

            log.debug(err, exc_info=True)

           print(traceback.format_exc())

            self._connect()

            return False

 

로그를 찍어보면

 

Traceback (most recent call last):

  File "c:\users\ansgy\appdata\local\programs\python\python38\lib\site-packages\s3_uploader.py", line 164, in upload_file

    self.s3.upload_file(

  File "c:\users\ansgy\appdata\local\programs\python\python38\lib\site-packages\boto3\s3\inject.py", line 143, in upload_file

    return transfer.upload_file(

  File "c:\users\ansgy\appdata\local\programs\python\python38\lib\site-packages\boto3\s3\transfer.py", line 300, in upload_file

    raise S3UploadFailedError(

boto3.exceptions.S3UploadFailedError: Failed to upload C:\Users\ansgy\\main\python\updateTest\pyu-data\deploy\keys.gz to robertchicken/robot_controller/keys.gz: An error occurred (AccessControlListNotSupported) when calling the PutObject operation: The bucket does not allow ACLs

 

아래와같이 발생합니다..

 

 

 

그래서 

 

C:\Users\ansgy\AppData\Local\Programs\Python\Python38\Lib\site-packages\boto3\s3\transfer.py 를 찾아들어가서 upload_file 를 아래처럼 로그를찍어보면

 

def upload_file(

        self, filename, bucket, key, callback=None, extra_args=None

    ):

        """Upload a file to an S3 object.

        Variants have also been injected into S3 client, Bucket and Object.

        You don't have to use S3Transfer.upload_file() directly.

        .. seealso::

            :py:meth:`S3.Client.upload_file`

            :py:meth:`S3.Client.upload_fileobj`

        """

        if not isinstance(filename, str):

            raise ValueError('Filename must be a string')

        subscribers = self._get_subscribers(callback)

        print('>>>>>>>>>>>>',filename,)

        print('>>>>>>>>>>>>', bucket)

        print('>>>>>>>>>>>>', key)

        print('>>>>>>>>>>>>', extra_args)



>>>>>>>>>>>> C:\Users\ansgy\\main\python\updateTest\pyu-data\deploy\app-win-1.0.0.zip

>>>>>>>>>>>> [bucket_name]

>>>>>>>>>>>>  [bucket_name]/app-win-1.0.0.zip

>>>>>>>>>>>> {'ACL': 'public-read'}

 

 

이 찍히네요

 

ACL 부분이 수상해서 None 처리해보니까  정상 배포되는것을 알수있엇습니다.

 

이것으로 S3 배포를 마칩니다

 

 

티스토리툴바