GH-49184: [CI] AMD64 macOS 15-intel Python 3 consistently times out

[tadeja]

Rationale for this change

Recent CI checks failing with the job AMD64 macOS 15-intel Python 3 being cancelled at 60 minutes.
The job has exceeded the maximum execution time of 1h0m0s

What changes are included in this PR?

Temporary timeout increase from 60 to 75 minutes for macOS Python 3 jobs (both ARM64 and Intel jobs as these have common setting.)

Are these changes tested?

To be tested on CI.

Are there any user-facing changes?

No.

• GitHub Issue: [CI] AMD64 macOS 15-intel Python 3 consistently times out #49184

[github-actions]

⚠️ GitHub issue #49184 has been automatically assigned in GitHub to PR creator.

[kou]

We need to change the macos job instead of the docker job:

diff --git a/.github/workflows/python.yml b/.github/workflows/python.yml
index bc7fe3cd68..e9db71a8c7 100644
--- a/.github/workflows/python.yml
+++ b/.github/workflows/python.yml
@@ -142,7 +142,7 @@ jobs:
     name: ${{ matrix.architecture }} macOS ${{ matrix.macos-version }} Python 3
     runs-on: macos-${{ matrix.macos-version }}
     if: ${{ !contains(github.event.pull_request.title, 'WIP') }}
-    timeout-minutes: 60
+    timeout-minutes: 75
     strategy:
       fail-fast: false
       matrix:

[tadeja]

Thanks, @kou! Updated.

[kou]

BTW, can we speed up macOS jobs...?

It seems that tests were finished in about 5min on Linux:

https://github.com/apache/arrow/actions/runs/21831237516/job/62989873777#step:6:6707

= 7836 passed, 253 skipped, 22 xfailed, 2 xpassed, 53 warnings in 247.63s (0:04:07) =

But tests were finished in about 30min on macOS:

https://github.com/apache/arrow/actions/runs/21831237516/job/62989873535#step:10:541

= 7527 passed, 573 skipped, 11 xfailed, 2 xpassed, 53 warnings in 1634.73s (0:27:14) =

Have you profiled tests on macOS?

[tadeja]

@kou speeding up jobs would be best, indeed!

I've been checking Build and why ccache almost never retrieves for macOS 15-intel - as it gets evicted due to the 10 GB repo-wide cache limit.
Cache not found for input keys: python-ccache-macos-15-intel-

I see just around 7 successful completions in February
12m example A and 4m example B with ccache

(Perhaps using sccache for macOS could be an option?)

I will check Test phase and profiling options next. For now I've added "--durations=20 -v" on pytest

Locally on M1 I get pytest finished in 86.44s (0:01:26) and the following test duration for comparison:

=========================================== slowest 20 durations ============================================
5.91s call     tests/test_fs.py::test_s3_options[builtin_pickle]
5.90s call     tests/test_fs.py::test_s3_options[cloudpickle]
4.67s call     tests/test_cython.py::test_cython_api
3.80s call     tests/test_extension_type.py::test_cpp_extension_in_python
3.76s call     tests/test_cython.py::test_visit_strings
2.95s call     tests/test_dataset.py::test_write_dataset_with_backpressure
1.66s call     tests/test_csv.py::TestThreadedCSVTableRead::test_cancellation
1.47s call     tests/test_fs.py::test_s3_real_aws_region_selection
1.08s call     tests/test_io.py::test_compression_level[zstd]
1.06s call     tests/test_csv.py::TestSerialCSVTableRead::test_cancellation
1.02s call     tests/test_pandas.py::test_is_data_frame_race_condition
0.97s call     tests/test_fs.py::test_s3_finalize
0.66s call     tests/parquet/test_metadata.py::test_table_large_metadata
0.64s call     tests/test_pandas.py::TestConvertMisc::test_threaded_conversion_multiprocess
0.60s call     tests/test_fs.py::test_s3fs_wrong_region
0.60s call     tests/test_fs.py::test_s3_finalize_region_resolver
0.58s call     tests/test_fs.py::test_concurrent_s3fs_init
0.50s call     tests/test_fs.py::test_s3_real_aws
0.48s call     tests/test_ipc.py::test_read_year_month_nano_interval
0.46s call     tests/test_pandas.py::test_threaded_pandas_import

[tadeja]

ARM64 macOS 14 Python 3 shows deterioration

============================= slowest 20 durations =============================
854.97s call     tests/test_pandas.py::test_nested_chunking_valid
84.06s call     tests/test_convert_builtin.py::test_string_too_large[ty1]
59.02s call     tests/test_convert_builtin.py::test_auto_chunking_binary_like
55.42s call     tests/test_convert_builtin.py::test_string_too_large[ty2]
54.07s call     tests/test_convert_builtin.py::test_large_binary_array[ty1]
46.97s call     tests/test_feather.py::test_chunked_binary_error_message
46.07s call     tests/test_convert_builtin.py::test_large_binary_array[ty0]
36.98s call     tests/test_pandas.py::TestConvertStringLikeTypes::test_bytes_exceed_2gb
30.02s call     tests/interchange/test_conversion.py::test_pyarrow_roundtrip_large_string
29.80s call     tests/test_convert_builtin.py::test_array_from_pylist_data_overflow
28.25s call     tests/test_convert_builtin.py::test_nested_auto_chunking[ty1-x]
26.66s call     tests/test_convert_builtin.py::test_nested_auto_chunking[ty0-x]
26.12s call     tests/parquet/test_parquet_writer.py::test_parquet_writer_chunk_size
25.14s call     tests/test_dataset.py::test_write_dataset_with_backpressure
24.57s call     tests/test_convert_builtin.py::test_string_too_large[ty0]
23.43s call     tests/test_array.py::test_list_child_overflow_to_chunked
22.41s call     tests/test_convert_builtin.py::test_auto_chunking_list_like
17.74s call     tests/test_pandas.py::TestConvertListTypes::test_auto_chunking_on_list_overflow
16.53s call     tests/test_pandas.py::TestConvertStringLikeTypes::test_auto_chunking_pandas_series_of_strings[x1]
15.80s call     tests/test_pandas.py::TestConvertStringLikeTypes::test_auto_chunking_pandas_series_of_strings[x0]

For ARM64 macOS 14 Python 3 durations didn't print yet as it got cancelled at [ 99%] this time within the new 75m timeout setting. I will try to run once more with 90m timeout setting.

[tadeja]

AMD64 macOS 15-intel Python 3 finally succeeds in 1h 16m 47s

============================= slowest 40 durations =============================
1712.00s call     tests/test_pandas.py::test_nested_chunking_valid
114.92s call     tests/test_convert_builtin.py::test_large_binary_array[ty0]
90.11s call     tests/interchange/test_conversion.py::test_pyarrow_roundtrip_large_string
76.32s call     tests/test_convert_builtin.py::test_large_binary_array[ty1]
73.84s call     tests/test_convert_builtin.py::test_auto_chunking_binary_like
54.73s call     tests/test_convert_builtin.py::test_string_too_large[ty1]
35.69s call     tests/test_feather.py::test_chunked_binary_error_message
35.54s call     tests/test_convert_builtin.py::test_nested_auto_chunking[ty0-x]
35.05s call     tests/test_convert_builtin.py::test_nested_auto_chunking[ty1-x]
30.69s call     tests/test_cython.py::test_cython_api
28.96s call     tests/parquet/test_parquet_writer.py::test_parquet_writer_chunk_size
27.06s call     tests/test_convert_builtin.py::test_array_from_pylist_data_overflow
26.29s call     tests/test_pandas.py::TestConvertStringLikeTypes::test_bytes_exceed_2gb
25.96s call     tests/test_array.py::test_list_child_overflow_to_chunked
24.99s call     tests/test_dataset.py::test_write_dataset_with_backpressure
22.27s call     tests/test_io.py::test_compression_level[zstd]
21.09s call     tests/test_pandas.py::TestConvertListTypes::test_auto_chunking_on_list_overflow
18.70s call     tests/test_cython.py::test_visit_strings
17.46s call     tests/test_convert_builtin.py::test_auto_chunking_list_like
16.43s call     tests/test_extension_type.py::test_cpp_extension_in_python
14.16s call     tests/parquet/test_data_types.py::test_large_binary_overflow
...
=========================== short test summary info ============================