Description
iCapital is looking to hire an experienced Python Developer with expertise in web scraping, automated portal interactions, and cloud-native deployment using AWS. The ideal candidate will have hands-on experience working with Playwright for browser automation, managing multi-factor authentication (MFA) flows, and deploying scalable scraping tasks via AWS Lambda and related services. This role will assist in architecting and building robust, secure, and scalable scraping solutions that interact with complex web applications and secured portals.
Responsibilities
- Assist with the design and implementation of advanced scraping solutions using Python, Playwright, and AWS services.
- Automate interactions with JavaScript-heavy and authentication-secured websites, including handling MFA, CAPTCHAs, and session or token-based login flows.
- Architect scraping pipelines using serverless AWS components such as Lambda, Step Functions, S3, CloudWatch, and Secrets Manager.
- Build systems that scale to support high volumes of data extraction with fault tolerance, retries, and intelligent logging.
- Integrate and manage complex workflows across multiple portals, APIs, and data sources.
- Contribute to architectural decisions, tooling, and best practices.
- Responsible for Code reviews and deployments.
Qualifications
- Bachelor’s or master’s degree in computer science, software engineering, or related field, or equivalent professional experience
- 10+ years of experience in Python development, with a strong focus on automation and data extraction
- Proven expertise in web scraping using tools like Playwright, Selenium, Scrapy, BeautifulSoup, and requests
- In-depth experience handling multi-step authentication flows, including multi-factor authentication (MFA), CAPTCHA solving, and session or cookie management
- Proficient in deploying and managing scraping workloads in AWS, particularly Lambda, S3, IAM, CloudWatch, and Secrets Manager
- Experience with asynchronous programming, headless browsers, and JavaScript-rendered content
- Solid understanding of web protocols (i.e. HTTP, HTTPS, cookies, headers), and the ability to reverse-engineer network calls and authentication mechanisms
- Comfortable working with APIs, JSON/XML, and data transformation
- Experience with CI or CD pipelines, Docker, and infrastructure as code (i.e. CloudFormation, Terraform)
- Familiar with data pipeline orchestration tools such as Apache Airflow or AWS Step Functions
- Prior experience scraping from secure or enterprise-level portals (i.e. financial, healthcare, legal)
- Background in data engineering or ETL workflows is a plus
- Exposure to Python testing frameworks and writing unit or integration tests
We believe the best ideas and innovation happen when we are together. Employees in this role will work in the office four days, with the flexibility to work remotely one day (Friday).
Benefits
iCapital offers a comprehensive benefits package that includes a total compensation program consisting of competitive salary, annual performance bonus, and equity for all full-time employees; healthcare with 100% employer-paid health and dental insurance; and generous paid time off (PTO).
For additional information on iCapital Network, please visit https://www.icapitalnetwork.com/about-us Twitter: @icapitalnetwork | LinkedIn: https://www.linkedin.com/company/icapital-network-inc
Similar jobs
Est. 165,000 USD
About the Role iCapital is looking to hire an experienced Python Developer with expertise in web scraping, automated portal interactions, and cloud-native deployment using AWS. The ideal candidate will have hands-on expe…
Est. 97,500 USD
About the Role iCapital is seeking an Associate Python Developer to join the growing Core Operations Technology team. This individual will have foundational experience in Python programming and a passion for building sca…
Est. 125,000 USD
About the Role iCapital is seeking a motivated and detail-oriented Associate Business Process Analyst, UiPath Developer to join our Intelligent Automation team. The ideal candidate will combine solid business analysis sk…
Job Title: Python Engineer About Us “Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companie…
Est. 125,000 USD
About the Role iCapital is seeking a motivated and detail-oriented Associate Business Process Analyst, UiPath Developer to join our Intelligent Automation team. The ideal candidate will combine solid business analysis sk…
Est. 117,500 USD
About the Role iCapital is looking to hire a Backend Engineer Associate or Assistant Vice President to join the Software Engineering team. This individual will play an integral role in designing and implementing systems…
Job Title: Java Python Full Stack Developer About Us “Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top…
Est. 150,000 USD
About the Role The Backend Engineer at iCapital will play an integral role in designing and implementing systems that power the iCapital platform. We are looking for engineers who can build APIs and applications that are…
Est. 115,000 USD
About the Role The Full Stack Engineer at iCapital will work end-to-end on new features, enhancements, and defect resolution on one or more of iCapital's software platforms. You will work closely with our Product Managem…
Est. 205,000 USD
About the Role The Platform Infrastructure team at iCapital plays a critical role in ensuring that both production and development environments operate smoothly, securely, and reliably. This role leverages advanced cloud…
Est. 140,000 USD
About the Role iCapital is seeking a strategic, hands-on, and results-driven Vice President Program Manager to join the Product Operations team, directly supporting the Pre-Investment vertical. This role will lead the pl…
Est. 65,000 USD
About the Role iCapital is looking to hire an Analyst to join its Platform Support team. The team provides service and technical support to its rapidly growing client base of financial advisors and individual investors,…
Est. 140,000 USD
About the Role The Full Stack Engineer at iCapital will work end-to-end on new features, enhancements, and defect resolution on one or more of iCapital's software platforms. This individual will work closely with our Pro…
Job Title: Python Developer About Us “Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Compani…
Est. 150,000 USD
About the Role The Backend Engineer at iCapital will play an integral role in designing and implementing systems that power the iCapital platform. We are looking for engineers who can build APIs and applications that are…
Est. 132,500 USD
About the Role iCapital’s Distributed Ledger Technology team is looking to hire a Senior Blockchain Backend Engineer to develop an industry-wide distributed ledger solution for the alternative investment ecosystem to fac…
Est. 80,000 EUR
About the Role iCapital is looking for a Full-Stack Engineer to join the iCapital Identity Platform team. This role will help define and deliver new features for one or more of iCapital's software platforms. You will wor…
Est. 55,000 EUR
About the Role iCapital is looking for a talented Data Analyst– Platform Solutions to join the International Platform Solutions team in Lisbon. The role encompasses supporting multiple projects focused on technical imple…
Est. 80,000 EUR
Senior Python / Agentic AI Engineer Capco: Where Innovation Meets Opportunity Capco is a global leader in consulting, uniquely positioned to drive innovation and transformation in the financial services industry. More th…
Est. 185,000 USD
About the Role iCapital is seeking a Vice President Artificial Intelligence Engineer to lead the design, development, and delivery of production-grade AI systems that drive measurable business outcomes across the firm. T…
Est. 72,250 USD
About the Role iCapital is seeking an Associate to join the Digital Marketing and Strategy team, reporting to the Assistant Vice President, Digital Marketing. This role is the execution engine for a portfolio of iCapital…
This role supports the ongoing delivery of agentic AI–enabled automation across document-heavy and operational workflows. The intent is to move critical automation work off a high-cost contractor and into a lower-cost, f…
Est. 175,000 USD
About the Role iCapital’s Distributed Ledger Technology team is looking to hire a Senior Blockchain Backend Engineer to develop an industry-wide distributed ledger solution for the alternative investment ecosystem to fac…
WorldQuant develops and deploys systematic financial strategies across a broad range of asset classes and global markets. We seek to produce high-quality predictive signals (alphas) through our proprietary research platf…
Job Title: AWS Support Engineer About Us “Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Com…
Est. 80,000 USD
About the Role iCapital is seeking a Client Delivery Specialist Associate to join the Automation team within the Data Solution department. This role will ensure a seamless flow of financial data for iCapital’s clients an…
Est. 80,000 USD
About the Role iCapital is looking for a Client Delivery Specialist Associate to join the Data Solutions team. This role will set up and maintain the financial software applications we utilize, assist clients with issues…
Est. 140,000 USD
About the Role iCapital is looking to hire a Client Delivery Vice President to join the Annuities Platform team. This role is responsible for leading the successful delivery of client implementations and ongoing client d…
Est. 120,000 USD
About the Role iCapital is seeking an Application Architect to lead the architecture, design, and modernization of the illustration related applications. This role serves as a key member of the Core Architecture Team, pa…
Est. 80,000 USD
About the Role iCapital is seeking a Client Delivery Specialist Associate to join the Automation team within the Data Solution department. This role will ensure a seamless flow of financial data for iCapital’s clients an…