a year ago
I´ve trying to deploy my scraping server, but the railway faild building e deploying
here is the log system and my dependencies in package json
I have authorization to scrapy, plus it's public data
"dependencies": {
"axios": "^1.6.8",
"dotenv": "^16.4.5",
"env-var": "^7.4.1",
"fs": "^0.0.1-security",
"jsdom": "^24.0.0",
"node-cron": "^3.0.3",
"node-schedule": "^2.1.1",
"nodemailer": "^6.9.13",
"openai": "^4.29.2",
"pg": "^8.11.3",
"playwright": "^1.42.1",
"prompt-sync": "^4.2.0",
"sequelize": "^6.37.1"
}
0 Replies
I can not recommend using Browserless enough, using it means you don't have to install Chrome and all of its needed dependencies into your container, and this will significantly speed up both the build and the publishing stage.
Browserless template here - https://railway.app/template/browserless
And here's my little example repo that shows you how to use browserless with playwright - https://github.com/brody192/playwright-example
but?
WSENDPOINTPLAYWRIGHT=${{Browserless.WSENDPOINTPLAYWRIGHT}}
yes that's provided by the template
nope, open your pre-existing project and click the + New
button and search for the template
yes, as long as you follow my instructions
looks good!
const browser = await playwright.chromium.connect(process.env.WSENDPOINTPLAYWRIGHT);
yep that looks right
you'd also want to remove your custom build command that tries to install chrome
build logs please
okay and show me your playwright.chromium code please
are you sure you arent calling chromium.launch elsewhere?
your code is still trying to launch chrome
yes you do, please read the readme
WSENDPOINTPLAYWRIGHT=${{Browserless.WSENDPOINTPLAYWRIGHT}}
yes, that needs to be set on your app's service
instead of typing out the name of the service you use i simply said app as a generic cover, i am talking about the service in the top left of this screenshot
what?
that would be an issue
theres no need for pay lol
i appreciate the offer though ❤️
well now you are self-hosting it, so you only have to pay railway's usage pricing
nah your english is fine
is your service not working?
then the logs are normal
look this line 'page.waitForTimeout: page.waitForSelector: page.waitForSelector: Target page, context or browser has been closed'
browserless closes its chromium browser when not in use to save on resources
but it was in 100% use, i didn't understand mt well i'm going to change the codes and test again
each call to chromium.connect would spawn another browser and then it would get closed, if your service works fine, then theres nothing to worry about
im not sure what "1/8 of the data" means
the browser is closing because they did what your code asked them to do
thats how browserless works
beacuse the page are open and the console show this issues: page.waitForTimeout: page.waitForTimeout: page.waitForTimeout: page.waitForTimeout: page.waitForTimeout: page.waitForTimeout: page.waitForTimeout: page.waitForSelector: page.waitForSelector: page.waitForSelector: page.waitForSelector: Target page, context or browser has been closed
at links (/app/pegarlinks_caixa.js:59:24)
at /app/async file:/app/test.js:23:29
okay but is that causing your app to not work correctly?
Yes, as it is a site that takes a long time to load there are waitforselector with very long timeouts
browserless has it's timeout set at 5 minutes, I can't see you needing more than that